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Executive Summary 

Understanding Public School Accountability 

Across the nation and in California in particular, it is impossible to ignore the increased 
emphasis on accountability and its impact on public school policies and practices. Account- 
ability policy has gained wide support among state policymakers, in response to concerns 
that public schools were failing to meet the needs of all students. California’s Public Schools 
Accountability Act (PSAA) holds schools and the educators within them accountable for 
meeting and sustaining high levels of student achievement. A system of standardized testing, 
curriculum standards, and rewards and sanctions provides the tools and motivation in an 
effort to improve school performance. 

State and federal policymakers continue to support accountability as an effective means to 
improve schools, encouraged by early indications of increased test scores. Surprisingly, there 
has been little research on local educators’ experiences with and responses to such reforms. 
This lack of research is striking, since teachers, principals, and superintendents are directly 
responsible for the implementation of accountability mandates, including administering tests, 
teaching to the state standards, and implementing state-approved curriculum packages. 

In an effort to understand teachers’ and administrators’ experiences with public school ac- 
countability, Policy Analysis for California Education (PACE), with the support of the Noyce 
Foundation, established the Educator Responses to Accountability Project (ERAP). During 
the 2002-2003 school year, we conducted in-depth interviews with educators in eight el- 
ementary schools across California. We spoke with teachers, principals, and district leaders, 
exploring such topics as their knowledge and understanding of recent accountability mandates, 
the impact of accountability on their classroom practices and their sense of professionalism, and 
their efforts to address inequities in student achievement within the context of accountability. 

Listening to Educators 

This report discusses key themes that have emerged from our conversations with educators, 
drawing on the voices of public school teachers, principals, and district administrators to 
present a portrait of accountability policies at the school level. We heard critiques as well as 
expressions of appreciation for recent state efforts, and we heard suggestions on ways to 
improve or supplement existing policies. 

Educators' responses to accountability are complex. 

Our interviews reveal that educators, and teachers in particular, are responding to account- 
ability in complex, and often contradictory ways. Teachers feel frustrated and overwhelmed 
by what they feel is a disproportionate amount of responsibility for accountability reform. 
Elowever, at the same time, they recognize the need for school accountability and appreciate 
certain aspects of California’s system. The standards component, for example, was seen as a 
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useful tool for teachers. Unlike the testing component, whose results often arrived too late in 
the school year to be of much use to teachers, or the rewards and sanctions element, which 
teachers felt was arbitrary and biased, standards elicited both positive and negative responses. 
Responses to accountability are simply not as uniform or unilateral as might be predicted. 

School districts play a significant role in how teachers experience accountability. 

Districts are often overlooked when it comes to accountability, because state policies target 
schools as the unit of change and do not hold districts directly accountable. This oversight is 
significant, given the district’s role in the implementation, and in many cases expansion, of 
state accountability policies. Three of the four districts visited have increased testing consid- 
erably, in terms of the amount of testing required and/or the grades that are expected to 
participate. Three districts have also created standards-based report cards or other account- 
ability measures used to determine both teachers’ and students’ understanding of curriculum 
standards. Only one district was notable for its relative lack of involvement, presenting an 
example of a more school-based approach to accountability. 

Teachers may experience unintended consequences of accountability. 

California’s system of accountability is grounded in the assumption that curriculum stan- 
dards and testing will focus teachers’ instruction and thus produce improved student out- 
comes. Our research confirms that the system is indeed influencing teachers’ work in elemen- 
tary school classrooms, at times producing unintended or potentially negative consequences. 
Testing and test preparation often displaces other instructional activities and leads to a nar- 
rowing of the curriculum. In addition, accountability pressures are causing teachers to feel 
they have less control over classroom decisions and diminished satisfaction with their work. 

Principals play a central role in how teachers experience accountability. 

Our research revealed that principals often played a pivotal role in how teachers experienced 
accountability policies. Principals either acted as a “buffer,” shielding teachers from test-score 
pressures, or as an added source of pressure for teachers, emphasizing the need to raise test 
scores and the school’s ranking. Principals’ attitude toward accountability and their leader- 
ship styles also influenced teachers’ experiences with district and state mandates. Principals’ 
emphasis on or avoidance of certain district and state instructional reforms, for example, was 
reflected in teachers’ acceptance or critiques. 

Accountability policies do not yet provide the necessary tools to address inequities. 

A common goal of accountability reform has been to draw attention to discrepancies in 
achievement across groups of students, and to hold schools responsible for addressing any 
inequities. Standardized test score data, disaggregated by racial, linguistic, and socioeconomic 
subgroups, serves as the mechanism for reform. Our research reveals that the path from test 
score publication and analysis to pedagogical change and academic improvement was not 
always clear. Moreover, educators were not convinced that accountability policies could ad- 
dress achievement gaps, pointing to flaws in the current system. Educators recognized that 
any effective solution lies beyond simply reviewing test scores, and requires a deeper investi- 
gation as to “why” inequities exist. 
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Improving upon California's System of Accountability 

The Educator Responses to Accountability Project has provided an opportunity to bring 
educators’ voices into education policy discussions, in an effort to illuminate the benefits and 
challenges of the system, and to expand policymakers’ understanding of accountability at the 
school level. Our conversations with educators also reveal aspects of the system, at both the 
state and district levels, that show potential for improvement. 

Ensure coherency and consistency in state accountability policy. 

Educators expressed confusion over the changing nature of California’s system of account- 
ability, including the calculation of API scores with different tests from year to year, and the 
availability and distribution of rewards. Although the PSAA has been successful in bringing 
attention to issues of student achievement, its clarity and understanding are essential to any 
sustained impact. The state must ensure that educators understand the policy’s guidelines, 
and feel they have the ability and capacity to meet its requirements. As federal mandates 
continue to overlap with California’s system, it is increasingly important the state make ef- 
forts to explain accountability policies to educators, parents, and students. 

Focus on building and sustaining capacity at all levels. 

State policy is based upon the assumption that most teachers and administrators have the 
capacity to address gaps in student achievement. Elowever, as our conversations reveal, many 
educators felt they did not possess the proper tools, including time, resources, and adminis- 
trative support, to systematically address issues of equity. 

Professional development is a key factor in improving educator capacity, particularly in the 
use and interpretation of data from statewide assessments. Principals and teachers need to be 
equipped with the necessary skills to decode data, with a focus on subgroup trends. Our 
research demonstrates that educators lack exposure to data as well as the time and skills to 
analyze data to inform instruction. As a result, educators are unable to achieve this primary 
goal of accountability, namely to address inequities through the analysis of disaggregated data. 

Finally, the state must consider the implications of holding all schools to the same expecta- 
tions while failing to address existing disparities in resources, materials, and funding. 

Recognize the integral role districts play in accountability. 

Currently, accountability policy assumes that individual schools are the primary locus of 
reform. While school-based reform is essential, our research shows that districts, through 
various programs and policies, have the power to facilitate or hinder school improvement. 
The state must recognize district influence in the implementation of accountability policy. As 
accountability demands continue to place pressure on schools, it is increasingly vital that 
schools are able to turn to their districts for support, guidance, and clarifications. 

Due to federal NCLB requirements, the state is now considering ways to hold districts ac- 
countable for the performance of their schools. Educators interviewed agreed that the state 
should provide a ranking or score for districts, similar to the Academic Performance Index 
(API) system for schools. Such a system should include a special focus on the lowest-perform- 
ing schools that have shown little to no signs of improvement. 
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Make strategic changes to the assessment and data components. 

While educators appreciated the state’s efforts to align its annual standardized test with state 
curriculum standards, with the recent shift from the SAT-9 to the CAT-6, they challenged the 
utility and fairness of relying on one assessment to measure a school’s success. In order to 
piece together a more coherent picture of student achievement, state and district leaders 
should also consider using ongoing diagnostic assessments to measure improvements in stu- 
dent achievement over a school year. Such tests would provide more immediate feedback, 
helping teachers to focus instruction and facilitate student learning. 

Although the state provides annual disaggregated data, few teachers are familiar with the data 
or are comfortable using it. Furthermore, teachers receive student test results too late in the 
academic year to be able to modify instruction. While this data does help illuminate achieve- 
ment gaps, it fails to aid teachers in making valuable changes during the course of the school 
year. Specific attention needs to be made to the analysis of data, disaggregated by student 
subgroup, as a basis for conversations around the identification and explanation of any ineq- 
uitable trends in achievement, and the strategizing of appropriate classroom responses. 



0 
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introduction 

Policy Context 

California, which had long prided itself on an excellent system of public education, was given 
a wake-up call in 1994 when its schools tied for last place on the National Assessment of 
Education Progress (NAEP). There was little doubt that California’s public school system was 
in desperate need of improvement. Its low test scores contributed to a strong sense of failure, 
and state policymakers were moved to respond to mounting concerns about the quality of 
California’s public schools. 

California’s former governor Gray Davis emphasized education as a top priority of his admin- 
istration. In 1999 the legislature approved major elements of the Public Schools Accountabil- 
ity Act (PSAA), centralizing school reform at the state level. PSAA incorporated a system of 
statewide testing, rewards and sanctions, and the recently created curriculum standards. This 
legislation was seen as an important first step in monitoring and improving student achieve- 
ment, allowing the state to measure each school with an achievement index, and offer assis- 
tance for underperforming schools or awards for schools making significant gains. 1 

PSAA was designed for transparency and objectivity, holding all schools and students to the 
same standards and exposing to the public the successes and failures of local schools. How- 
ever, Davis’ PSAA faced immediate challenges. Teachers’ unions and education associations 
criticized the legislation for lacking input from school-level educators. Moreover, California’s 
recent economic growth took a sharp downturn, eventually forcing budget cuts across the 
state, including several PSAA components. 

The last two years have brought further adjustments to California’s system of accountability. 
The introduction of federal No Child Left Behind (NCLB) legislation has resulted in addi- 
tional layers of reporting and sanctions. California’s Democratic governor Davis was recalled, 
replaced by Republican governor Schwarzenegger. Given this political shift, it is likely that 
California’s system of accountability will face further adjustments in the coming years. 

Educator Responses to Accountability Project 

Amidst these shifts in policy and school reform at the state and federal levels, researchers at 
PACE have asked, “How are educators experiencing and responding to accountability at the 
school level?” At the heart of any system of accountability are educators — teachers, princi- 
pals, and superintendents responsible for its successful implementation. Yet, despite 
policymakers’ continued focus on school accountability throughout California and across 
the nation, very little is known about school-level actors’ experiences. 

With the support of the Noyce Foundation, PACE established the Educator Responses to 
Accountability Project (ERAP) in an effort to provide a much-needed focus on teachers’ and 
administrators’ experiences with the current system of accountability. ERAP is one of the first 
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studies in California to look at such issues as the impact of accountability on classroom 
teaching and learning, educators’ sense of professionalism, and the shifting role of the district 
in negotiating state mandates . 2 

ERAP also considers the potential for a system of accountability to address issues of equity, 
including ethnic, socioeconomic, and linguistic diversity in California public schools. While 
the assumption behind California’s system of accountability is to improve education for all 
students, the processes for achieving that goal are not always apparent. Thus, it is critical to exam- 
ine the challenges and possibilities of addressing issues of equity within a study of accountability. 

The voices of teachers, principals, and superintendents are rarely heard during discussions of 
education policy development. Yet, as we discovered, designing policy without educator in- 
put may result in a disconnect between the initial goals of a policy and the realities of school- 
level implementation. Despite early criticisms, there are strong indications that school ac- 
countability will remain a top priority on the state and national agenda. Thus, in an effort to 
improve upon current conditions, it is vital that we consider the experiences of educators. 

This report aims to give policymakers a better understanding of how accountability reform 
plays out inside our schools. As California aspires to create and sustain high levels of student 
achievement, it is now more crucial than ever that policymakers have access to information that 
illuminates the intense challenges as well as potential rewards of public school accountability. 



FIGURE 1 . Educator Responses to Accountability Project: Distribution of 2003 API Scores by Student 
Socioeconomic Indicator 
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Methodology 

The Educator Responses to Accountability Project provided an opportunity to listen to teachers 
and administrators, and to foster communication between educators and policymakers. 
Through a qualitative case study approach, we sought to document educators’ experiences 
with California’s public school accountability system. This research is unique in its ability to 
add the necessary layers of complexity to existing quantitative data, which often uses student 
outcome measurements to assess accountability’s impact on achievement. While those mea- 
surements are important to consider, they provide little understanding of the impact of ac- 
countability on school and classroom practices and the lives of educators and students (Bogdan 
& Biklen, 1998). 



Research Sites 

ERAP focused on educators’ experiences in elemen- 
tary schools, due to a lack of research on account- 
ability at that level as well as an interest in the im- 
pact of accountability during the earlier grades of 
state testing. The study focused on eight elemen- 
tary schools across the state — two schools each in 
four districts. In order to capture the diversity of 
California’s public schools, the sites represented a 
range of student populations, community charac- 
teristics, and student achievement levels. 3 

Data Collection 

During the 2002-2003 school year, each of the 
eight schools was visited once for an intensive three- 
day period, and again, when possible, for a half- 
day. Three research team members participated in 
each of the intensive visits; one senior member of 
the team conducted the half-day visits. 



FIGURE 2. ERAP Teacher Experience* 




*Does not include "no response" 



The three-day intensive visits included: 

■ Interviews with all kindergarten, second, and fourth grade teachers, 

■ Observations of a kindergarten, second, and fourth grade classroom, 

■ One focus group with other key personnel such as the assistant principal, Title I 
specialists, curriculum coaches, and special education teachers, 

■ An interview with the principal, and 

■ An interview with the superintendent or other district leaders. 

In addition, the visit included two informational meetings with faculty and staff: an overview of 
the project at the beginning of our visit, and a summary of initial findings at the end of the visit. 

The half-day visits were used to supplement the intensive visits. When the brief visit occurred 
in the fall, it served as an introduction to the project; when it occurred in the spring, it served 
as a follow-up. These visits allowed us to view changes that occurred in schools’ policies and 
experiences throughout the year. 
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Until recently, 
research on 
educators' 
experiences of 
accountability in 
California has 
been largely 
nonexistent. 



Interviews and focus groups followed a semi-structured protocol, covering a set of themes 
while allowing participants to shape the interview based on their unique experiences. 4 Inter- 
views and focus groups lasted approximately one-hour each and were audio-taped. 

Teacher and administrator interviews and focus groups focused on the following themes: 1) 
the participants professional background, 2) the school and district context, 3) the participant’s 
experiences with and responses to accountability, and 4) equity issues. In total, we inter- 
viewed eight principals, four district administrators, and 65 teachers individually, and con- 
ducted seven focus groups of approximately 4-6 teachers each. 

Alongside interviews with school-level educators, PACE interviewed eight key state -level 
policymakers, including members of Governor Davis’ staff, the California Department of 
Education (CDE), the Senate Education Committee, and a statewide education association. 
Each policymaker was intimately involved with the design of California’s system of account- 
ability, and thus provided critical insight into the historical and political forces that shaped 
the reforms that were ultimately implemented. Each offered a definition of accountability 
and an understanding of the rationale behind the system. 5 

Despite the acknowledged shortcomings, policymakers uniformly praised California’s system 
of accountability as an important step towards improving education. They were also asked 
what they would like to know from local educators about the implementation of the PSAA. 
Policymakers expressed a great deal of interest in educators’ views on how accountability 
reforms played out at the school level and ways the program could be improved. 

Data Analysis 

The research team met weekly throughout the processes of data collection and analysis to 
debrief on site visits and discuss emerging themes. Team members prepared extensive field notes 
for each interview, which were shared with the entire team. They transcribed and coded all 
interviews and focus group discussions using a qualitative research software package. 
The thematic coding scheme was modified as necessary to accommodate emerging themes 
and complexities. 

The findings presented in this report represent teachers’ and administrators’ experiences across 
a variety of school contexts. Despite differences in school and district leadership, student 
populations, and achievement levels, our interviews revealed common perspectives regarding 
the impact of the state’s accountability system on educators’ professional lives and their 
efforts to improve student achievement. 

Review of Research Findings 

Until recently, research on educators’ experiences of accountability in California has been 
largely nonexistent, in part due to the fact that the state’s system is relatively young. ERAP 
has provided the necessary insight into both the challenges and benefits of California’s policy 
at the school level. 

Chapters 2-6 present findings from our research in schools, organized by themes that emerged 
from our interviews with over 100 educators throughout the state. While the themes dis- 
cussed in this report represent the broad range of perspectives, it is important to recognize 
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State Policymakers' Views on Accountability 

Our conversations with state-level policymakers revealed several common assumptions and 
critiques of California's system of accountability, as exemplified by the following quotes: 

"Ready, fire, aim." 

The policymakers interviewed reminded us that PSAA was designed under intense pres- 
sure and made public in less than three months of development. As a result, educators 
implemented a complex accountability system with many details, including the testing 
system, still in transition. Furthermore, the system's goals did not always mesh with its 
design. For example, California included "world-class" curriculum standards as a key com- 
ponent of accountability, which were intended to guide and inform teaching and learn- 
ing. Yet the state's original Academic Performance Index was based on results from the 
SAT-9, a nationally-normed test that is not aligned with California's curriculum stan- 
dards. The PSAA subsequently tied rewards and sanctions to the API, essentially sending 
educators a mixed message. While acknowledging that the state needed more time to 
diagnosis and correct flaws, several policymakers noted that the political environment 
at the time encouraged expediency. 

"If you build it, they will come.” 

PSAA was based on the assumption that educators would be motivated to reach and 
sustain high levels of student achievement by rewards (monetary, public recognition) 
and sanctions (low-performance labeling, threat of school takeover). Although 
policymakers agreed that teacher buy-in would be a crucial factor, time constraints did 
not allow for extensive dialogue with school-level actors. Instead, educators were 
simply expected to acknowledge the inherent value of an accountability system and 
make necessary adjustments in their practices. The state perceived its role as creating 
a framework to measure and reward growth, while educators were responsible for 
achieving that growth. 

"The system sunshines the equity issue.” 

While a few policymakers mentioned curriculum standards as a necessary component 
to ensure equity, the majority of respondents believed that the achievement data would 
lead to more equitable practices and outcomes. The publication of API scores would 
highlight achievement gaps, ensuring that schools could no longer ignore low-perform- 
ing subgroups. While the state would provide extra funding for a select group of schools 
(through ll/USP), policymakers felt that the disaggregated data provided educators 
adequate capacity and tools to address disparities and close achievement gaps. 



that educators’ responses are not always unilateral, and in many cases are quite complex. 
Chapter 2 explores this issue of complexity, challenging the notion that teachers are unilater- 
ally resistant to accountability. Teachers recognize positive aspects of the system, including 
renewed attention to issues of student achievement and the introduction or reemphasis of 
curriculum standards. 
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However, educators also offer criticisms of the system of accountability. Chapter 4 discusses 
educators’ concerns about the impacts of accountability on curriculum and instruction, as 
well as their sense of professional satisfaction. Teachers, for example, report spending an 
increased amount of time on tested subjects (typically math and language arts), and a 
decreased amount of time on non-tested subjects. Such concerns have been echoed in studies 
of teachers’ experiences with accountability in other states (Herman, 1 990; Jones et. ah, 1 999; 
Stecher et. al., 1999; Taylor et. ah, 2003). 

Educators also report an increased sense of pressure as a result of the state system of account- 
ability. This pressure stems from efforts to make sense of and meet the requirements of a 
complicated system, with what they feel is a lack of additional support or resources. Teachers 
and administrators were particularly concerned about overwhelming demands placed on their 
time, as a result of increased paperwork and reporting, as well as an increase in the amount of 
testing and material to be covered in the classroom. 

The additional pressure and regulatory nature of accountability caused many teachers to 
report a decreased satisfaction in their work, coinciding with a loss of autonomy over their 
pedagogy and curriculum and a sense of not being treated as professionals. Similarly, princi- 
pals and district administrators commented on the overwhelming nature of their work, as 
they struggle to balance efforts to raise test scores and close achievement gaps with mundane 
but equally challenging administrative tasks such as fixing a leaky roof. The education profes- 
sion has clearly shifted in this age of accountability. 

While ERAP focuses on teachers, our research reminds us that their experiences with ac- 
countability are mediated by district and principal leadership (Firestone & Mayrowetz, 2000; 
Herman, 1990; Noble & Smith, 1994; Simon, Foley & Passantino, 1998; Fairman & Firestone, 
2001). Chapter 3 examines the role of the district, as superintendents may choose to empha- 
size certain aspects of the accountability system, or in some cases, implement additional 
layers of accountability. Chapter 5 examines the role of the principal as either a buffer or a 
source of pressure for teachers. Principals’ interpretation of state and district mandates, for 
example, may influence teachers’ acceptance of or resistance to accountability reforms. 

Chapter 6 offers a broader analysis of one of the primary assumptions of accountability, 
namely that the system will facilitate educators’ efforts to close gaps in student achievement. 
Policymakers had hoped that the presentation of disaggregated achievement data would highlight 
any inequitable patterns, and subsequent subgroup growth targets would motivate educators 
to make necessary adjustments to instruction. While the policy has raised educators’ awareness 
of inequities, educators are concerned that it stops short of helping schools build capacity, in 
terms of time and resources, to design and implement solutions to achievement gaps. 

In light of educators’ responses to accountability, both positive and negative, Chapter 7 pro- 
vides a discussion of the policy’s current strengths as well as areas for improvement. Our 
interviews with teachers and administrators reveal aspects of the system that are working. 
They also suggest opportunities to adjust the system, at the state and district levels, to better 
equip educators to improve student achievement. It is our hope that this report will inspire 
continued discussions among educators and policymakers, in an effort to build upon 
California’s efforts to improve education for all students. 
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Complex Responses to Accountability 



While public school accountability continues to gain national prominence and federal sup- 
port, concerns have been raised that such policies may negatively impact teachers and their 
work in the classroom (Darling-Hammond 1994; Elmore 2002). In California in particular, 
there was significant concern that the PSAA would lead to a lack of control, causing frus- 
trated teachers to leave the profession (CTA, 2002). Yet, our interviews reveal that teachers 
are responding to accountability in complex, and often contradictory ways. Teachers do feel 
frustrated and overwhelmed by what they feel is a disproportionate amount of responsibility 
for accountability reform. However, at the same time, teachers recognized the need for school 
accountability, and appreciated certain aspects of California’s system. 

The standards component in particular was seen as a useful tool for teachers. Unlike the 
testing component, whose results often came too late for use, and the rewards/sanctions 
element, which teachers felt was arbitrary and biased, standards elicited both positive and 
negative responses. The notion of curriculum standards has provoked criticism for limiting 
teachers’ freedom and control (Meier 2000; Nathan 2000), and our interviews with teachers 
did confirm those concerns. Yet those same teachers also felt that standards offered numerous 
benefits that positively impacted their work in the classroom. In essence, responses to 
accountability are not as uniform or unilateral as predicted. 

Accountability as a Signaling Tool 

There was general agreement that any efforts to improve education should include holding 
teachers responsible for their actions. While no teacher expressed that they directly needed 
accountability, teachers referred to others who were “not doing what they needed to do.” 
While it was not clear what “doing it” entailed, there was a general sense that some teachers 
were failing to meet the needs of their students. One Beech Elementary School teacher 
echoed this sentiment, noting: 

We have some students who can’t read. They went through the system and 
they don’t have the skills. So people are saying, “Well how can you go through 
12 or 13 years of school and still can’t read, or read at a basic level?” And so 
things had to be happen where there were people more accountable. 

Many teachers believed that under previous policies, unmotivated and incompetent teachers 
were allowed to remain in the classroom. A system of accountability was seen as the first step 
in sustaining improvements in the teaching profession. 

By drawing attention to inefficiencies within the teaching profession, accountability was 
essentially a signaling tool, indicating that teachers had to refocus on “core” subject and 
content matter. Several teachers commented that accountability gave them direction and 
focus. For instance, a veteran teacher at Maple Elementary School commented that although 
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Teachers felt 
standards played 
a direct role in 
motivating them to 
expect more from 
their students. 



she did not have much enthusiasm for accountability when it first started, she grew to feel 
that it was refreshing because it enabled her “to see what you teach in a new way.” By exam- 
ining her teaching, she exposed her students to more difficult concepts and was motivated to 
present material in greater depth. 

A teacher at Poplar Elementary School echoed this sentiment, explaining that accountability 
had motivated her to become more serious and committed in her work. For instance, she 
started placing more emphasis on problem-solving skills. In short, the focus on accountabil- 
ity led some teachers to reexamine their instruction and its impact on student achievement. 

Excess Focus on Teachers 

While teachers felt that accountability reforms drew attention to unproductive teachers, this 
attention was also seen as somewhat unfair or misguided. California’s system of accountabil- 
ity, with its emphasis on annual testing and API scores, placed a great deal of pressure on 
teachers to produce outcomes. Teachers felt they were usually the ones blamed for any occur- 
rences of low performance at their schools. They advocated instead for a more holistic 
approach to student achievement, pointing out that administrators, parents, and students 
also play a role in a school’s successes or failures. 

Indeed, several teachers raised the importance of parental involvement to student achieve- 
ment, noting that the current system of accountability failed to involve parents in any 
substantial way. This failure was seen as a huge oversight, as one teacher observed: 

To hold teachers solely accountable for remedying all of the ills of society is 
unconscionable. It’s not realistic. I think it’s important for there to be 
accountability and for there to be systems in place to provide that account- 
ability, but there has got to be a way of balancing it, of not putting this 
threat over teachers. (Teacher, Maple) 

Another teacher remarked that although accountability “starts with parents,” it was unclear 
how parents were being held responsible. Accountability placed unreasonable expectations 
on teachers, with limited support from parents and administrators. 



Benefits of Curriculum Standards 

Standards positively affect teacher expectations. 

In the most basic sense, standards are a direct articulation of what students are expected to 
learn. Standards may also be seen as the first step in reducing educational disparities among 
high-poverty and underrepresented students, by holding all students to the same expecta- 
tions. Therefore, it is noteworthy that we heard numerous accounts from teachers who felt 
standards played a direct role in motivating them to expect more from their students. For 
example, one teacher required her first graders to write more, another teacher exposed his 
students to more difficult math concepts, and a bilingual teacher became more conscious of 
motivating his students to read in English. One teacher at Beech described the direct effect 
standards had upon her expectations: 

A few years ago, if the children were first-graders, they’d write two or three 
sentences. Where now, I found myself [thinking] , “Okay. Well, two or three 
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sentences is good in September, but by now you should be writing a couple 
of pages.” I found myself requiring them to do more in a way that’s appro- 
priate for their ability. 

Although standards cannot guarantee an “equal” education, it is clear that they play an important 
role in providing teachers with clear goals and expectations for all students across the state. 

Standards provide direction and focus. 

Another strong argument for standards was that they provide a tangible direction and focus on 
specific material and concepts, a clear and detailed path for teaching: “I have a goal at the end now, 
I know what these things are going to look like, I know I have to go down this road” (Teacher, 
Sycamore Elementary School). The idea of knowing what to teach was a huge benefit, as one 
Redwood Elementary School teacher explained: “ [I can] look at what I am supposed to teach, 
what students are going to be tested on, and know what is coming up.” It is important to note that 
support for standards does not imply that teachers were previously “lost,” but simply that this 
aspect of the system of accountability provided a means to refine and improve their teaching. 

Specifically, teachers reported that standards helped them with lesson planning, enabling 
them to review and incorporate key concepts. One teacher emphasized that planning was an 
easier task, as she divided her school year on how she was going to meet the standards. 
Another teacher recalled the difficulty of teaching in a pre-standard environment, noting: 

The standards are the best thing they ever did. Because, I was just newly 
teaching and I went into a classroom with nothing. There were some books. 

I’m supposed to teach them what? The first year I started, the standards 
weren’t out. So I went to the teachers in the grade above me and said, “What 
do you want the kids to know?” And then I went to the other teachers and 
asked, “What are you teaching your kids?” I tried to figure it out in 
between, and I mean, that was difficult for me. (Teacher, Maple) 

The inability to properly plan and prepare was seen as a barrier to effective teaching, and 
standards played a direct role in alleviating this burden. 

Along with a sense of clarity, teachers reported that standards facilitated reflection. Reflection 
plays an essential role in teaching, helping teachers to constantly grow and search for new 
ways to reach students. Yet, it is often difficult to reflect without benchmark tools. One 
experienced teacher at Aspen Elementary School noted that with standards, 

You look and you think, “Alright, here is my timeline and this is what I need 
to teach, and I want to get them to a certain point.” So as a teacher, it makes 
me look at myself, and every June, I reflect back on the year, what went 
right, what didn’t, and how I can change it. Because I think as educators, we 
can always change what we’re doing and we can always improve on things. 

Furthermore, reflection entails the sharing of strategies, successes, and failures. Because grades 
now follow uniform standards, teachers were able to partake in grade-level reflection and 
planning. A group of teachers at Poplar reported that they meet before the start of the school 
year to review strategies to incorporate standards. Looking at previous test scores in conjunc- 
tion with the standards, they focused on key areas of concern to determine which standards 
were the most essential for student achievement. 



Teachers across 
all sites expressed 
concern that the 
current grade-level 
standards were 
developmental^ 
inappropriate. 
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How Ongoing Standards-Based Assessment 
Enhances Accountability 



The majority of educators interviewed 
expressed concern about the inability of 
the state's standardized test data to in- 
fluence classroom practices in meaning- 
ful ways. While the state has recently 
made efforts to align the tests to curricu- 
lum standards, scores are still not released 
until the end of the summer. This delay 
prevents teachers from using test scores 
to inform and modify their instruction 
during the course of the school year. 

In fact, our study revealed that teachers 
value data if it is provided in a clear and 
timely manner. This strategic use of data 
was evident at Poplar Elementary, where 
teachers supported a practical and valu- 
able ongoing classroom assessment pro- 
gram. The program is based upon the no- 
tion that in order for students to under- 
stand advanced concepts, they must first 
have basic skills. Unlike state testing, the 
assessment program evaluates student 
achievement in smaller, more useful, 
increments. Since it provides immediate 
scores and feedback for each student, 
teachers have multiple opportunities to 
reteach and reemphasize key standards. 
As one teacher noted: 

We're using assessments in 
smaller increments to see student 
growth instead of, "Oh the next 
grade level, what do we do with 
them?" And we're really trying 
to assess students and change 
our instruction while they're still 
in our classroom. 

In addition, the program is standards- 
based, providing compatibility, rather than 
competition, with the state's system of 
curriculum standards and testing. One 
teacher explained that the program, 
"made me stop and think, Why am I teach- 
ing this? Why am I doing this? Now, we 
are stopping and thinking, What is this 



teaching? What standard am I covering 
now by doing this project?" Moreover, 
teachers could use these test scores as early 
indicators for the end-of-year state test, 
and adjust daily lessons plans as necessary. 

Several teachers commented that they now 
have a common language to use for student 
assessment, allowing teachers to meet on 
a regular basis to discuss patterns and areas 
of concern. As one teacher explained: 

We look at testing scores. That data 
is analyzed for each individual stu- 
dent and any kind of patterns are 
analyzed, any trends we might 
see, inadequacies in certain areas. 

And then we decide what our 
plan of attack is going to be. 

There was a noticeable sense of collabo- 
ration at Poplar, as teachers offered critical 
feedback and shared advice and strategies. 

The success of the assessment program 
at Poplar can be attributed in part to its 
useful and timely data and its compatibility 
with the state system. It is also important 
to note that the program's implementa- 
tion was initiated by teachers, and enjoyed 
strong support among school and district 
administrators. Indeed, the support of the 
principal was central to the program's suc- 
cess, as one teacher remarked: 

He [the principal] went to our 
weeklong training with us. It made 
a big difference in terms of my 
buy-in to the whole project be- 
cause he was willing to be there, 
he didn't have to do that.. .so that 
made a big difference. 

Poplar provides an interesting model of 
assessment and data analysis. With strong 
teacher and administrator buy-in, educators 
recognize the usefulness of ongoing assess- 
ments that allow for immediate reflection 
and adjustments to classroom instruction. 
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Critiques of Curriculum Standards 

Standards place unrealistic demands upon teachers and students. 

Numerous teachers remarked that state curriculum standards placed unrealistic demands 
upon their profession. They felt that teachers were fighting a losing battle because the standards 
were simply too high for the majority of their students. One teacher at Maple shared her daily 
frustration of trying to balance the standards with the academic needs of her students: 

I think that the standards they’re trying to meet are fair for kids who come 
into this school prepared to learn. But the standards keep increasing as the 
years go by, and the kids who are not at grade-level early on, in kindergar- 
ten, first, or second-grade really struggle. Even at my grade level, we have 
some kids who just can’t read the curriculum [material] . And we try to catch 
them up, but at the same time I have 1 8 other kids and often 1 0 or 1 5 of 
them are way beyond their grade-level. 

Second grade teachers interviewed, for example, often struggled to teach students the com- 
plex concept of time, or advanced math properties of multiplication and division, which 
many felt were inappropriate at that grade level. 

Indeed, teachers across all sites expressed concern that the current grade -level standards were 
“developmentally inappropriate.” Such concerns caused a pedagogical dilemma for many 
teachers — should they teach material for which students were not ready? And if not, what 
were the ramifications of not shoring up basic concepts, such as addition and subtraction, 
before moving on to multiplication? For example, a Maple teacher reported that a fourth 
grade standard requires students to master algebra with expressions and variables. With her 
best students struggling to master these advanced concepts, she felt conflicted over the appro- 
priateness of the required material. 

The inappropriate nature of standards led to strong feelings of resentment among many 
teachers interviewed, who disliked being told what to teach by policymakers who were per- 
ceived as removed from classroom realities and having little or no teaching experience. Teach- 
ers attributed the unrealistic nature of California’s standards to a lack of teacher input during 
the development process, and many felt the state failed to address the difficultly of imple- 
menting standards. One discouraged teaches remarked: 

Sometimes I’d like to see them [the state] come in to our classrooms and 
teach some of the programs, and [then] let us know what they really think 
about the program. Do you think that these students are going to progress 
as fast as, you know, we expect them to do on that state standard? (Teacher, 

Spruce Elementary School) 

Despite their overall support of state curriculum standards, teachers were nonetheless frus- 
trated by their not having a voice in policy decisions. 

Teachers were also concerned about the sheer magnitude of standards, forcing them to adopt 
a “breadth over depth” approach to teaching. The large number of topics forced them to 
move through the curriculum before students had a chance to master concepts. One teacher 
told us how she was able to just teach “a little bit of this” before she had to move on to “a little 
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bit of that.” (Teacher, Aspen) Another commented that teachers would better “serve our 
students if we had a more narrow field of what they asked us to do.” (Teacher, Poplar) 

Despite their support of standards as a policy reform, teachers ultimately felt frustrated with 
the realities of implementation. As one teacher explained, “The biggest challenge is trying to 
cover everything and having students grasp what they are learning in the amount of time we 
have to do it in, in the quantity of time and quantity of material to teach. It’s a losing battle.” 
(Teacher, Aspen) These concerns call into question the fundamental rationale behind cur- 
riculum standards as an aspect of accountability. What is the purpose of standards if teachers 
are unable to keep pace with rigorous demands ? 6 

Teachers’ responses to California’s system of accountability reflect the complexity of the 
issues involved. The teachers interviewed were not resistant to the idea of being held account- 
able. They supported certain aspects of accountability such as curriculum standards, which 
facilitated, not hindered, their efforts to improve student achievement. They also offered 
critiques of those and other aspects, such as standardized test data which did not meet their 
needs. Finally, teachers provided examples of alternative accountability efforts, such as the 
use of an ongoing classroom assessment program, which suggest adjustments to the current 
state system. 
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The Role of the District 

Districts are often overlooked when it comes to accountability. Most state policies, California’s 
included, target schools as the unit of change and do not hold districts directly accountable. 

However, recent research has revealed that, in fact, districts do matter. Districts often respond 
in one of two ways: they either buffer or pay little interest to state policy (Firestone and 
Fairman, 1998), or they add an additional layer of accountability by mandating their own 
assessments and developing their own performance incentives (Chrispeels, 1997; Goertz et 
al., 1998). Our research primarily supports the latter finding. As one teacher in the Tech 
Valley district explained, “It definitely feels like there’s a lot more assessment coming from 
the district and that they’re writing a lot more of their own accountability measures” (Teacher, 

Aspen). Furthermore, when asked who they felt accountable to, many teachers cited the 
district above the state. 

Three of the four districts visited have expanded testing considerably, both in terms of the 
amount of testing required and the grades expected to participate. Three have also created 
either standards-based report cards or other accountability measures in an effort to track 
teachers’ use and students’ understanding of the standards. In addition, two have mandated 
curriculum packages that are aligned with the standards, and developed accompanying pac- 
ing plans that dictate the order and timeframe in which concepts are taught and tested. 7 

However, one district was noticeable for its lack of activity in comparison to the other dis- 
tricts, especially since more than half of its schools did not meet their state growth targets. 8 It 
is difficult to conclude from our data whether the district saw additional accountability 
measures as unnecessary or if it was intentionally buffering state policy. Possible reasons for 
the district’s behavior will be discussed later in this chapter. 

Additional Layers of Accountability 

District testing 

Teachers in three of the districts talked extensively about testing required by their districts, in 
addition to the SAT-9. In North and South City districts, where Open Court and Scott 
Foresman are mandated, teachers were expected to administer Open Court tests approxi- 
mately every six weeks and Scott Foresman tests every quarter. The South City district also 
mandated its own math test, administered four times a year, as well as an annual perfor- 
mance writing assignment. It is notable that unlike the SAT-9, which is only mandated by 
the state for grades 2-5, North and South City require district tests for all grades, including 
kindergarten. South City also requires its first-graders to take the SAT-9, even though their 
scores are not sent to the state. 

Teachers in Tech Valley have also experienced an increase in district testing. In grades 
K-2, teachers are expected to administer a language assessment, which measures students’ 
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reading ability four times a year. The district also requires students to take performance 
based assessments in math, language arts, science, and social studies. 

Only two teachers interviewed in the Central Plains district mentioned tests required by the 
district. One teacher talked about an annual writing assessment, which is given to all stu- 
dents; another referred to an end-of-the-year district test for students in kindergarten and 
first grade. Instead most teachers discussed the assessment programs used at their specific 
schools. While teachers in the Central Plains district test their students just as frequently as 
the other districts, they do not see assessment as a burden but rather as a tool that informs 
their instruction. This is likely attributable to the fact that these programs were chosen by 
teachers, not mandated by the district. 9 

Teachers in the Tech Valley and South City districts told us that increased testing has become 
a source of pressure in the classroom. Due to the number of required district tests, some 
teachers are administering at least one and as many as three tests a week. As a teacher from 
South City remarked, 

There is so much testing that we have to do with the kids that I would say 
a month of the year is spent on testing the kids. And our kids are in school 
for only eight months. So what is that,12%? And that is a long time that 
you cannot really be teaching them because you are testing them. 
(Teacher, Sycamore) 

Interestingly, this stands in stark contrast to the amount of time South City district leaders 
estimate its teachers spend on testing. One district administrator agreed that “some of the 
rhetoric that you hear regarding the testing burden makes it sound like it could be a very large 
number.” According to his calculations, however, just “a little over 1% of elementary stu- 
dents’ time is spent in testing for assessment.” While it is impossible to determine which 
percentage is more accurate, it is clear that the district may not have a sense of the impact of 
the “testing burden” on teachers’ work in the classroom. 

Because of the immediacy and frequency of district tests, some teachers explicitly 
identified the district as the entity to which they feel they are being held accountable. 
This is not altogether surprising, since who teachers see themselves being held account- 
able to is often influenced by what they see themselves being held accountable for 
(Abelman & Elmore, 1999). The teachers who named the district as the agent of ac- 
countability also identified district-mandated curriculum tests as what they were being 
held accountable for: 

The district is [holding us accountable] because we were told that all our 
test scores go to the district and there is some guy at the district who looks at 
all the test scores. . ..I think many of the teachers feel the same way as I do, 

[they feel] a lot of the pressures, a lot of the stuff that is going on that the 
district is requiring us to do. (Teacher, Sycamore) 

Interestingly, as implied in the quote above, some teachers felt that they were not necessarily 
being held accountable for their students’ performance on the test, but simply for the fact 
that the tests were administered. As one teacher in South City explained, 
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District pacing plans 

Teachers in both the South City and North City districts also identified district pacing plans, 
which accompany mandated curriculum packages and prescribe the order and time frame in 
which the lessons and tests are given, as a source of pressure. Many teachers complained that they 
were restrictive and unrealistic and tried to cover too much material in a short period of time: 

Because of the testing now. ..not anyone of us probably [are] right on the 
target date for where we’re supposed to be on the calendar so... it’s hurry 
up. . .because you’re already behind. They [the district] don’t allow time for 
the assessments when they schedule your calendar. . . .But I’m behind schedule 
and they’re going to do the next assessment. . .so you could see the madness of 
that. If you start acting like a robot and. . .following it to the millimeter you 
could probably pull that off but at what expense to kids? (Teacher, Maple) 

Teachers in both districts also expressed concern that the pacing plans did not allow enough 
time to reteach material, and that they sometimes had to administer assessments before cov- 
ering all the tested concepts: 

This year we are so bogged down. . .it is not good for the kids to have all these 
things. You have to teach Open Court for three hours and you have to be on 
a certain story, and you have to teach one math lesson a day. And if your kids 
don’t get it you are supposed to move on when you can’t, and then you get so 
behind that they have to take a test that they are not ready for. (Teacher, Beech) 

Pressure to keep up with the pacing plans has also led to a narrowing of the curriculum as 
teachers focus primarily on language arts and math at the expense of other subject areas or 
interdisciplinary projects: 

Impossibility is scheduled and you try to figure it out. How do you do 1 5 
minutes minimum of Hampton Brown for English language development? 

And two-and-a-half hours of Open Court a day? And a minimum of one 
hour of math a day? Plus the music, the art, the social studies, the science? I 
challenge anyone in the district to show where that could be possible with a 
scripted program that doesn’t overlap. I used to do interdisciplinary things, 
but you can’t do that with scripted programs. (Teacher, Redwood) 



We have tests. . .that must be given to our kids and they must be turned into 
the district. So all the paperwork has to be done. That is where the major 
accountability comes in — if you are not complying and getting the scores in 
on time. (Teacher, Sycamore) 

Similarly, a teacher in Tech Valley noted that teachers were expected to administer and grade 
the district assessments in math and writing themselves, but were not required to turn the 
scores into the district. 

The district’s role in mandating additional testing clearly had an impact on how teachers 
perceived the usefulness of those assessments. District-mandated tests were generally seen as 
a source of pressure for teachers, rather than a means to improve teaching. Additional testing 
chosen by teachers, however, was seen as more useful, given teachers’ engagement in policy 
decisions and the timeliness of that data. 
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District placing plans further exacerbated the time crunch teachers experience in the class- 
room; moreover, they raise concerns about the quality of student learning when teachers feel 
pressure to move on before students master the material. 

District monitoring 

The two districts using Open Court applied additional pressure by sending district personnel 
to schools to monitor program implementation and instruction. Teachers sometimes referred 
to these individuals as the Open Court “police.” One teacher at Redwood in North City who 
had been openly critical of Open Court in school and district meetings, told us that the 
superintendent visited her room three times in four months to make sure she was following 
the curriculum. In addition, one teacher at Sycamore in South City was told by the Open 
Court coach to stop using reading groups in her classrooms, since the curriculum package 
only allows for whole-class instruction. 

Several teachers at Willow Elementary School in Tech Valley also talked about district moni- 
toring of test scores and direct pressure they felt from the superintendent. When their stu- 
dents performed poorly on the SAT-9, the superintendent visited the school to berate them 
for their students’ test scores. However, the teachers added that the superintendent also praised 
the teachers when their scores improved the following year. 

The amount of direct pressure teachers felt from the district was mediated by the school’s 
academic performance. For example, in North City none of the teachers at Maple, with an 
API score of 858, identified the district as a source of pressure. One teacher there said that 
people from the district had stopped visiting, and now one of the teachers at the school 
simply collects the Open Court test scores and sends them to the district office. However, 
teachers at Redwood, also located in North City, but with an API score of 655 and an II/USP 
designation, expressed resentment over the way in which the curriculum was mandated by 
the district and how the district monitored instruction. 

Standards-based district accountability measures 

The South City district has developed additional accountability measures designed to help 
teachers focus on instruction and the use of standards in the classroom. For example, the 
district does walk-throughs at the schools, during which the superintendent, principal and / 
or other administrators and teachers visit classrooms and talk to students and teachers. They 
observe how a teacher questions, probes, and leads class discussions using the standards, and 
the degree to which students can articulate the standards that they are learning. In addition to 
incorporating standards “language” in the classroom, the district also promotes the use of 
other district “learning principles” such as setting clear expectations and encouraging aca- 
demic rigor. 

The Tech Valley district issues its own version of curriculum standards, distilling the state 
standards into “essential” district standards. Tech Valley, as well as the Central Plains district, 
also implemented the use of standards-based report cards, thus bringing further emphasis to 
the curriculum standards component of the state accountability system. Teachers at Willow 
in Tech Valley were overwhelmed by the length of the report card, which required teachers to 
describe a student’s performance on each standard. Despite the district’s efforts to limit the 
standards, teachers still felt overwhelmed by the additional requirements associated with this 
aspect of accountability. 



E0 ERAP 





CHAPTER 3 



As with additional district testing, the effectiveness of these accountability measures is tied to 
the degree to which they allow teachers to reflect upon and improve their teaching practices. 
Without this connection, these measures simply create additional work which teachers may 
eventually grow to resent. 

Taking a Less Active Approach 

Teachers in Central Plains rarely mentioned the role of the district in discussions of testing or 
accountability. They most often talked about the SAT-9, understood as a state-mandated test, 
and the assessment programs required within their schools. Unlike the other three districts, 
the Central Plains district has developed few accountability measures to supplement those of 
the state. They appear to have taken a more reactionary approach, addressing problems as 
they arise rather than trying to anticipate and prevent them. 

For example, when Spruce Elementary had allowed too many waiver exemptions for the 
SAT-9, the district did not inform the principal of the state’s policy on the number of 
students required to take the test. As a result, Spruce did not receive an API score that year 
and was reprimanded by the state. Both principals we interviewed in Central Plains have 
taken a more proactive approach in educating themselves and their teachers about state 
accountability policies. 10 

The Central Plains district’s lack of activity can be attributed to several factors. First, the 
district may not have the resources or capacity to develop their own assessments or account- 
ability measures, given its relatively small size. One principal described the district as “too 
busy and overwhelmed,” and has taken it upon himself to support his teachers in preparing 
for the state Standardized Testing and Reporting assessments. Smaller districts may be at a 
disadvantage in comparison to large districts that have the resources necessary to facilitate 
policy implementation, and thus may require additional support from the state in order to 
meet accountability demands (Flannaway & Kimball, 2001; Fairman & Firestone, 2001). 

Second, the district may have wanted to see how the state policy would unfold before re- 
sponding. Recently, the district has begun showing signs of moving in a similar direction as 
the other districts, taking on a more active role in accountability. During the 2002-2003 
school year, a few teachers at Spruce and Poplar piloted the Floughton Mifflin reading pro- 
gram, slated to be implemented throughout the district the following fall. Whether the dis- 
trict plans to mandate additional testing requirements or an accompanying pacing plan is still 
unclear. A few teachers at Spruce told us that the district is also planning a district-wide 
expansion of the popular assessment program used at Poplar. Thus, it appears as if the Cen- 
tral Plains district may be developing its own accountability system similar to those already 
underway in other California districts. 

Our research confirms what others have found, namely that districts play a key role in the 
implementation, and in many cases enhancement, of California’s system of accountability. 
Three out of our four districts took an active role in ensuring schools’ compliance with state 
accountability mandates, through the development of additional testing and standards-based 
measures. The Central Plains district appears to be taking on a more active role as well. With 
increasing state mandates and the overlap of NCLB requirements, more districts are likely to 
expand their role in school policy to meet the demands of accountability. 
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Unintended Consequences of Accountability 



Previous research has documented the effects of state accountability systems, and high stakes 
testing in particular, on elementary school curricula, teachers’ work, and student learning in 
Kentucky, Maryland, North Carolina, and Ohio. This research confirms that testing causes 
elementary school teachers to increase the time spent teaching tested subjects, and to decrease 
the time spent teaching non-tested subjects such as science and social studies (Jones et ah, 
1999; Stecher and Barron, 1999). In addition, teachers report increased time spent teaching 
directly to tests (Smith, 1991; Whitford and Jones, 2000). 

Teachers find high-stakes testing stressful and believe it negatively affects their students (Kubow 
& Debard, 2000). States’ efforts to motivate improvement may be undermined by teachers’ 
perception that testing is not a valid measure of teaching and learning, and their resentment 
in shouldering an unfair burden of responsibility (Min trap, 2003). Our study confirms and 
extends these findings by documenting elementary school teachers’ experiences with account- 
ability in California. 

California’s system of accountability is grounded in the assumption that curriculum stan- 
dards, testing, and evaluation of schools will focus teachers’ instruction and thus produce 
improved student outcomes. Our research confirms that this system is indeed influencing 
teachers’ work in many elementary school classrooms, at times producing unintended or 
potentially negative consequences. Specifically, testing and test preparation may displace other 
instructional activities and lead to a narrowing of the curriculum. In addition, accountability 
may negatively impact teachers’ work, ultimately causing teachers to feel less control over 
classroom decisions and diminished satisfaction with their work. 

Negative Impacts on Curriculum and Instruction 

The teachers we interviewed expressed a great deal of concern about the negative impact of 
accountability reforms on their classroom practices and the material covered. Increased state 
and district testing has resulted in teachers devoting more time to test preparation and 
administration. The tests’ focus on math and language arts, coupled with district-mandated 
curriculum packages in those subject areas, has reduced the time teachers can spend covering 
other subject areas, including science, social studies, and the arts. 

At the same time, teachers have struggled to incorporate a vast array of state curriculum 
standards, forcing them to choose between “breadth versus depth,” as discussed in Chapter 2. 
These various pressures of accountability ultimately have left students with a more ex- 
pansive yet superficial exposure to math and language arts, and minimal exposure to 
other subjects. 
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Testing and test preparation displace other instructional activities. 

While the state test only occurs during one week in the spring, each of the schools we visited 
administered additional tests as a result of state and district accountability policies. Conse- 
quently, many teachers report that frequent testing takes up time that would otherwise be 
spent teaching: 

[Considering] what’s required from the district, and from the state, it is 
almost like we should sit down and pick and choose, and come up with 
some kind of schedule where we are not always bombarding the students 
with this test and that test.... I just feel it takes away from instructional 
time. (Teacher, Willow) 

Beyond the time spent actually administering tests, teachers also experienced pressure to 
prepare students for those tests. It is important to note that teachers often distinguished test 
preparation activities as separate from or in addition to regular content area instruction. For 
example, a teacher at Beech Elementary explained, 

I’m doing more test preparation with the students because it’s not that they’re 
not capable; some students are. But if they’re not translating their knowl- 
edge in the classroom onto the test-taking themes, then it’s not really show- 
ing what they can do. So after I learned that, I said wow, I’m going to really 
work with my students more, and prepare them with more test prep so that 
they can be successful. 

Consequently, test preparation has become another activity that teachers must fit into their 
already limited instructional time with students. 

One potential consequence of constrained instructional time coupled with high stakes test- 
ing is that teachers will begin teaching directly to the tests. Approximately one in five teachers 
reported that they teach explicitly to tests. Additional teachers reported that they are aware of 
the phenomenon at their school, but did not comment on their own practice. Some teachers 
were concerned that teaching to tests would contribute to the narrowing of the curriculum to 
tested subjects. Others worried that teaching to the test would force teachers to cover more 
content but with less depth: 

You do not have time because you know that test is coming up any day now 
and they have to know this, this, and this. And if you introduce it to them, 
it will be better than if they have never seen it at all. So it fosters just skim- 
ming the water as opposed to going down deep. (Teacher, Sycamore) 

Narrowing of the Curriculum to Tested Subjects 

Two competing tensions have emerged from recent accountability reforms in California. On 
the one hand, state and district accountability measures have directed teachers to address a 
broader range of curriculum standards within the same amount of instructional time. On the 
other hand, at the time of this research, the state’s evaluation mechanism, the SAT-9, focused 
only on mathematics and language arts. 11 Not surprisingly, many teachers reported that they 
respond to the more immediate pressure of raising achievement scores by narrowing instruc- 
tion to focus on tested subjects. 
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Indeed, the testing, standards, and mandated curriculum components have all pushed teach- 
ers to focus on tested subjects at the expense of other content areas. The majority of class- 
room teachers reported that they spent very little time teaching science, social studies, or fine 
arts because they must emphasize math and language arts instruction to meet state or district 
level accountability demands. In the words of one teacher, 

There’s so much they expect you to do that it’s science that doesn’t get taught, 
art doesn’t get taught, you know. The things that cause children to love 
school and learn on their own are being cut. (Teacher, Aspen) 

Many teachers believe that they simply do not have enough time in the classroom with stu- 
dents to teach all content areas. Given time constraints, it is not surprising that they choose 
to emphasize the tested subjects that will form the basis for evaluating student, teacher, and 
school performance. 

The fact that language arts and math were the only tested subjects sent a clear, albeit unspo- 
ken, message to teachers about which subjects they should emphasize in the classroom. A 
teacher at Maple explained, “I look at people and I see that things don’t happen unless they’re 
tested... people teach what’s on the test. And if we get held accountable for teaching social 
studies and sciences, then they will be taught.” Teachers were more likely to report the nar- 
rowing of the curriculum in schools with district-mandated curriculum packages. In con- 
junction with program implementation, districts designed pacing plans for teachers, indicat- 
ing what lessons they were expected to cover daily. One teacher described her frustration with 
her district’s pacing plan saying, “If a program takes three hours long, well, what about the 
other things you have to do?” (Teacher, Beech) 

As mentioned in the previous chapter, district-mandated curriculum packages in math and 
language arts further contributed to an emphasis on those subjects at the expense of others. 

Within the context of limited time and the realities of accountability demands, teachers have 
been forced to make judgments about the relative value of each subject based on its inclusion 
or exclusion on standardized tests or in district mandates. 

Teachers voiced a number of concerns stemming from the narrowing of the curriculum. 

Many spoke of their inability to pursue student interests in the classroom through science, 
social studies, and art projects, and the negative impact this had on student engagement and 
motivation. One teacher at Redwood explained, 

I feel like I have less opportunity to be creative. It’s really hard to just find 
something kids like, and it’s hard to take it and run with it. You know? 

Because you have to be on a certain page or you have to be in a certain 
story, or you have to be ready for the SAT-9 or CAT-6 or whatever. 

(Teacher, Redwood) 

In addition, teachers feared that the exclusion of creative components of the curriculum took 
away an avenue for some students to experience success in school. A teacher at Aspen explained: 

Music will be cut you know, when for a lot of my kids that may be the one spot 
where they get to shine. One place where they get to have something they’re good 
at, or to be the best, because they might not be a good reader or good at math. 
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Hour Accountability is impacting Kindergarten 

Kindergarten is more complicated than it used to be. ..My recollections of 
kindergarten were clay, and games, and beads.. .Now, we're reading by the end 
of kindergarten. (Teacher, Willow Elementary) 

Many of the kindergarten teachers in our study commented on the changing nature of 
kindergarten in recent years, and the role accountability pressures have played in this 
shift. They felt the curriculum is much more academic as a result of the new standards 
and focuses less on students' social and physical development. For example, a third of 
the teachers interviewed reported that they feel as if they are teaching first grade 
now rather than kindergarten. As a teacher from Poplar Elementary explained, 

The standards from first grade have been pushed onto kindergarten, and we're 
supposed to teach them to read in kindergarten now.. .From when I started 
[teaching] 17 years ago, it's a big change. 

Teachers noted that while the expectations have changed, what kindergarteners are 
ready for has not. They voiced concern that the standards were developmental^ inap- 
propriate and that the expectations were too demanding and potentially detrimental 
to students' future academic development: 

The pressure now is to do more academic [work].. .[but] the kids are still devel- 
opmentally in the same place where they were.. .And for some of these kids, 
you are stepping over all of their developmental foundation things. To teach 
them something that they are not ready for. (Teacher, Redwood) 

Some teachers also expressed concern over the amount of testing required at the 
kindergarten level. Two-thirds of the kindergarten teachers mentioned a noticeable increase 
in testing over the past few years. While teachers are not expected to administer the 
annual state standardized test, they are often required to give district assessments, 
such as the Open Court and Scott Foresman tests or the Language Observation Survey (LOS): 

When I was a younger teacher in the district... we were just getting [one] stan- 
dardized test.. ..we thought that was really bad for kindergarteners [laughs]. 

And that's nothing compared to what we have now. It's like we took all these 
steps backwards rather than forward as far as I'm concerned with the testing 
issue. (Teacher, Aspen) 

As a result of these raised expectations, now more than ever before kindergarten 
teachers see themselves as the bridge between home and school. Many teachers com- 
mented on the importance of educating not only the students, but also the parents, 
so that they can reinforce classroom learning at home. Ultimately, accountability has 
brought kindergarten teachers a greater sense of responsibility for laying down the 
foundation for future academic success: 

I'm not doing the state testing in kindergarten. So I don't have that kind of 
pressure, but it starts in second-grade and I know that if I haven't taught 
certain things in kindergarten, that probably by the time the kids get to second- 
grade and take that test, they won't be prepared for it. So I feel responsible in 
kindergarten that I need to.. .build my part in the foundation. (Teacher, Willow) 
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Negative impacts on Teachers' Work and Professionalism 

Teachers report a disproportionate amount of pressure. 

One of the assumptions underlying the state accountability system is that giving teachers 
more explicit instructional guidelines and materials — such as content standards and curricu- 
lum packages — will focus and improve teachers’ instructional activities, and in turn result in 
improved student outcomes. Not surprisingly, teachers feel more pressure due to greater scru- 
tiny of teacher and student performance. The majority of teachers interviewed reported feel- 
ing higher levels of pressure that can be directly attributed to the accountability system. 
Sources of performance pressure include frequent testing, public reporting of test scores, and 
punitive aspects of the system such as labeling schools low-performing. Here’s how one teacher 
described the dilemma: 

You know there’s just a lot more pressure. We’re published in the paper. I mean, 
everything is a mark on what you look like and it’s never the children, it’s never 
the parents, it’s what are you doing. It’s your [the teacher’s] fault. Your school isn’t 
scoring, you’re school isn’t doing well. What is your problem? (Teacher, Aspen) 

Policymakers may argue that pressure is a desirable component of a system that demands 
more from teachers. However, most respondents felt that the constant pressure directed at 
teachers was not motivating. A teacher at Spruce explained, 

So the only thing that’s being addressed right now is the teachers. And it 
seems like teachers become very defensive about it and that’s not the right 
atmosphere for change and improvement. 

In fact, this Sycamore Elementary teacher suggested that the added pressure may be having 
the exact opposite effect on motivation: 

I think it makes the teachers feel bad because no matter what we do, it is just 
not good enough. And to be judged on that is very frustrating. It almost 
makes one want to not do this anymore. 

Ultimately, teachers resented what felt like a disproportionate amount of blame directed at 
them when students did not perform well on tests. “I think in general teachers are whipping 
boys for all of this,” said one Redwood teacher. 

In addition to the effects on teacher motivation, pressure resulting from the state’s account- 
ability mandates may exacerbate other negative consequences of the accountability system. 
For instance, pressure to improve student performance on standardized tests may exacerbate 
constraints on instructional time and the narrowing of the curriculum, as teachers feel com- 
pelled to emphasize test preparation or teach to the test. In addition, stress may negatively 
impact the relationship between teachers and students. When asked if the system of account- 
ability had changed her relationship with her students, a teacher at Sycamore told us, 

I am now more business-like and less, I mean you know as a parent you fool 
around with your kids and let them explore ideas, which I used to do with kids. 

They would come up with a question and we would talk about it more. And 
now it’s, “I am sorry children. We can’t talk about this now because I have 
this lesson I have to teach you.” So it kind of stifles the children’s curiosity. 
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Teachers report a loss of autonomy and professional satisfaction. 

Teachers reported that certain aspects of state and district accountability are a source of pressure, 
not motivation or assistance. Time pressure resulting from the number of content standards, 
frequent testing, and mandated curriculum packages, combined with performance pressure 
originating from testing, public reporting of test scores, and potential consequences for fail- 
ure to improve, together have serious implications for teachers’ work and students’ educa- 
tional experiences. In addition, our data suggests that accountability has caused a reduction 
in teacher discretion over classroom level decisions and a decrease in professional satisfaction. 

Many components of the state accountability system place tighter external controls over school 
policies and practices; consequently, teachers feel less control over their work in the classroom. 
The vast majority of teacher interviews touched on the issue of autonomy under accountability: 

It’s almost to the point where a teacher’s scripted as to exactly what they’re 
supposed to say and teach, what page you’re supposed to be on, a certain 
stage of the book, and what books you can use. (Teacher, Poplar) 

Some proponents of the accountability system argue that the loss of autonomy is not an 
altogether negative consequence of a system that seeks greater uniformity in teacher practice 
with the goal of more consistent positive student outcomes. Indeed, a stated goal of California’s 
accountability system was to limit teachers’ coverage to “important content” (California 
Department of Education, 1998). However, this teacher at Redwood reminds us how loss of 
autonomy can discourage teacher engagement and performance: 

I think maybe it [the system of accountability] has affected me negatively. I 
always think that somebody is breathing down my neck. And so if I deviate. . . 
if I say, we are going to plant some seeds today and put Open Court on the 
shelf. I kind of look over my shoulder to see if anybody heard me say to the 
kids that that is what we are doing. Or if I read this book instead of what the 
Open Court is telling me to do. I think it is a discomfort. It has not been 
anything that has enhanced my teaching or driven it in a positive way. 

Our research indicates that many teachers feel constrained in their ability to teach because of 
California’s system of accountability. As a consequence, teachers have experienced a loss of 
professional satisfaction. 

Everything has changed now. Um. . .the pressure. I am trying to be so pro- 
fessional about not having the stress and the constant pressure affect the 
classroom climate. It just took all the fun out of teaching. (Teacher, Redwood) 

Indeed, several teachers noted a loss of joy or creativity in their teaching, and the negative 
impact of that loss on their students: 

Everything we have to do we are so bogged down, there is absolutely no 
creativity in teaching anymore, and because of it our children are suffering. 

(Teacher, Beech) 

The narrowing of the curriculum, constant time and performance pressures, loss of autonomy, 
and negative effects on students have caused teachers to derive less satisfaction from their 
work. Erosion of professional satisfaction may have serious consequences, as professional 
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satisfaction is strongly related to commitment to teaching (Fresko et al„ 1997). The preva- 
lence of these themes throughout our sample suggests that this is a situation to be monitored; 
if loss of autonomy and professional satisfaction continue to escalate, teachers may be dis- 
couraged from remaining in the classroom. Consequently, erosion of satisfaction among 
California’s teachers may have serious implications for the state’s ability to staff classrooms 
with high quality, committed teachers. 
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The Role of the Principal 



Researchers have long pointed to the importance of school leaders and principals in imple- 
menting education reform (Fullan, 2001; Glickman, 1993; Senge, 2000), even in the context 
of state-driven accountability systems and state-mandated testing (Firestone, et al., 2001a, 
Smith, 1991b). While state and district mandates do have an impact on teachers’ work and 
instructional choices, principals can have just as much, if not more, impact on teachers’ 
understanding and implementation of accountability reforms (Firestone, et. ah, 2001b; 
Herman, 1990). In our study, we found that principals often played a pivotal role in how 
teachers experienced California accountability measures. 

In schools with low test-scores or a low-performing label, principals either acted as “buffers,” 
shielding teachers from pressures related to test-scores, or as an added source of pressure for 
teachers, by emphasizing the need to raise students’ test scores and the school’s API score. 
Principals’ attitude toward accountability and their leadership styles also affected how teach- 
ers experienced district and state accountability. Principals’ emphasis on or avoidance of cer- 
tain district and state instructional reforms, for instance, was reflected in their teachers’ con- 
cerns and practices related to accountability. Moreover, teachers seemed most accepting of 
reforms when they felt principals had engaged them in decision-making around the imple- 
mentation of accountability mandates at the school site. 



Principals: Buffer or Source of Pressure? 

Every principal in our study said they felt pressure related to accountability — usually in terms 
of raising test scores or enforcing new directives from the district. Some principals passed this 
pressure on to their teachers, however, while others worked as a “buffer,” shielding teachers 
from district and state pressures. The contrast between principals as buffers and principals 
who created additional pressure was most apparent in schools with low API scores, since 
those schools tended to be under the most external pressure to raise student achievement. 

For instance, during the year of our study, Willow and Aspen, two schools in the Tech Valley 
district, were in danger of having to send out letters informing parents of their status as 
“failing” schools. In both cases the schools had achieved this status based on the failure of one 
student subpopulation to meet its API targets. 12 

At Willow, where the principal focused energy and attention on standards-based reform and teachers’ 
instructional strategies, teachers reported feeling little pressure to improve test scores, and the pressure 
they did feel they attributed to the district rather than to their principal. As one teacher explained, 

[The principal] takes a lot of heat off of us. Certainly he’s getting a lot of 
pressure from the district and outside sources. . ..he sort of feels like it’s his 
role as principal to buffer us from that negativity. . ..he tries to keep us moving 
forward in a positive way instead of getting bogged down. 



Principals often 
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While one teacher recalled feeling pressured by a part-time assistant principal to improve her 
students’ test scores, most teachers we interviewed felt that the focus at Willow was on 
instruction and standards versus testing, and felt supported rather than pressured by the 
principal to improve. 

Willow teachers cited the principal’s effort to create a schedule that conformed to their needs, 
his encouragement of teachers to work together as grade-level teams, and his implementation 
of site-based decision-making as examples of how he actively supported them. The principal 
explained that he gave teachers more control and authority in making decisions affecting 
classroom instruction and curriculum under “the assumption that they’ll make good deci- 
sions.” He felt that this assumption was pivotal if accountability was to be meaningful for 
teachers. If teachers were going to be held responsible for what goes on at the school site, he 
explained, then “they have to have the authority to make [the necessary] changes.” Teachers’ 
statements in interviews and focus groups reflected this perspective. “We are definitely allowed 
to be professionals in the classroom... our principal believes in us,” one teacher remarked. 

While the principal at Willow buffered his teachers from testing pressures by focusing on 
instruction and training, the principal at Tech Valley’s Aspen, although very well-liked by his 
teachers, seemed to add to accountability pressures. Aspen’s principal viewed himself as 
protecting his teachers from district demands, yet teachers at the school felt enormous 
pressure to maintain their relatively high test-scores, and more often than not they cited the 
principal as the source of that pressure. 

For instance, some teachers referred to the principal as “test-driven,” particularly the kinder- 
garten teachers who had explicitly been told by the principal that they needed to raise their 
students’ scores on district tests. Teachers often identified the pressure they were feeling to 
raise test scores as coming from the district, characterizing the district as “breathing down” 
their necks, but it was the principal who reportedly conveyed the district’s message to them. 
Indeed Aspen’s principal acknowledged that he stressed testing and the school’s API with 
teachers, explaining that his attitude was to accept the state’s accountability system and 
encourage his teachers to do well within it. Whether or not he agreed with the system, he told 
teachers, “It is here and we have to do it, do it well, do it right, and do it the best you can.” 

Some teachers at Aspen cited the principal’s “competitive” personality in explaining the pres- 
sure they were feeling regarding test scores. Stated one teacher, “He wants us to be the 
best. . .that’s the bottom line. The best of all the schools in our district, in our state.” While 
Aspen teachers appreciated the high expectations set for them and their students, and for the 
most part felt supported by their principal, his emphasis on test scores and API rankings 
added additional accountability-related pressure to teachers’ day-to-day work at this school. 
As the above examples demonstrate, two schools under similar testing pressures and the same 
district directives have experienced accountability quite differently through different 
principal leadership. 



Principals' Attitudes Shape Instructional Response. 

The importance of the principal’s role in how teachers experienced California’s system of 
accountability went beyond adding to or shielding teachers from pressures to raise test scores. 
Our study found that principals’ attitudes toward specific accountability-related reforms also 
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impacted teachers. For example, schools in the same district at times implemented district- 
level accountability measures very differently based on their principals’ perceptions of the 
district directives. 

As discussed in Chapter 3, in South City, the school district had mandated curriculum pack- 
ages, pacing plans, and unit assessments along with an instructional approach centered around 
learning principles and standards-based language. However, the two South City schools in 
our study experienced these reforms quite differently as a result of the principals’ differing 
perspectives. The principal at Beech, who had worked at the district office in developing its 
instructional program prior to becoming a site administrator, described herself as well-aligned 
with the district and actively worked with her teachers to implement the district’s new 
reforms. At Sycamore, on the other hand, the principal was openly skeptical of the district’s 
new programs, and focused instead on being sensitive to the school’s history of strong teacher 
leadership and site autonomy. 

These two principals’ attitudes towards district accountability requirements were reflected in 
their teachers’ experiences. When asked about the various aspects of accountability, teachers 
at Beech often mentioned classroom walk-throughs and standards-based language along 
with state- and district-mandated testing. One teacher noted her new principal’s focus 
on accountability: 

[Our former principal] didn’t really care too much about exactly what the 
district wanted. Whereas our principal now is very much “This is what the 
district wants and this is what we have to do.” 

Moreover, teachers at Beech tended to associate their new principal with accountability. Even 
when they knew that the programs were coming from the district, they saw them as part of 
their principal’s agenda. 

Few teachers at Sycamore, on the other hand, mentioned classroom walk-throughs in their 
interviews, and none mentioned the instructional learning principles or standards-based 
language when asked about accountability. Instead, Sycamore teachers tended to speak of 
the district-mandated curriculum packages and accompanying tests. They did not associ- 
ate accountability at all with their principal, who they saw as a “hands-off” leader. As one 
teacher explained: 

The principal, he’s very nice.. . .it’s not like he makes you do a lot of extra 
things or anything, but. . . everybody is in the same boat. We all have to do 
it so it isn’t something he can help us out with. It is just something that 
everybody has to do. 

This teacher’s portrayal of the principal as disconnected from the district’s accountability 
reforms and somewhat powerless to impact change at the school site reflected the principal’s 
own view of himself as having to follow orders. He described the principal’s role in account- 
ability as one of being “...good soldiers — whether we necessarily agree or not, we do carry 
out the edicts and the dictates.” Sycamore’s principal tried to support the instructional 
programs already in place at the school by not emphasizing additional district rules and 
programs. While this did not constitute adequate support for those who were strongly 
opposed to the district’s mandates, teachers appreciated the principal’s buffering efforts: 
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[The principal] filters a lot of it, which is why I think we still have some 
buy-in. He really tries to, because he knows how independent we are as a staff. 

Interestingly, teachers at both schools had mixed views about their principals’ instructional 
leadership. While some teachers at Sycamore appreciated the principal’s efforts to minimize 
directives, others expressed frustration that the principal was not more of an instructional 
leader and did not push teachers to improve their practice and look carefully at student 
achievement. Similarly at Beech, while some teachers appreciated the principal’s attention to 
classroom instruction and pedagogy, others felt that in stressing accountability she ignored 
other important aspects of teaching, such as classroom management. At both schools, how- 
ever, whether they liked their principals’ leadership styles or not, teachers’ understanding of 
and experiences with accountability reflected their principals’ commitments and styles as 
instructional leaders. 

Involving Teachers in Accountability Decisions 

Teachers overall were most accepting of accountability-related reforms when they felt they 
had played a role in their adoption at the school-site. For example, Poplar Elementary in the 
Central Plains district had not implemented many district-wide reforms at the time of our 
study, yet teachers there spoke positively about the instructional reforms they had chosen to 
implement. Poplar teachers were particularly supportive of a standards-based assessment pro- 
gram they had adopted the year before, and the way that it brought student achievement to 
the center of their staff discussion . 13 While they emphasized that much of their support for 
the program stemmed from the fact that its adoption had been a teacher-initiated “grass- 
roots” process, they also credited the principal in particular for promoting teacher-led 
accountability at Poplar: 

[The principal’s] philosophy has been ... if we all decide to do something, 
then it will be successful, if we all buy into whatever we decide. 

This atmosphere created greater buy-in at the school site, and made teachers feel more 
accountable for the program’s success. Poplar’s principal agreed that he promoted teacher 
buy-in as a critical part of accountability, explaining, “We know, [if] teachers and people 
believe in what they’re doing then it’s much more likely that’s going to happen.” Teachers saw 
a clear relationship between feeling empowered by the principal and feeling accountable: 

[The principal] really takes into account where we want to go we’re all 

deciding what direction this school will take. Of course that’s empowering 
and makes you want to do a better job because you feel like you have a say in 
what you’re doing. 

Similarly, teachers at Willow, where the principal had implemented site-based decision-making, 
tended to view accountability in terms of their responsibility to one another and their students. 
While schools in Tech Valley were somewhat more constrained by district initiatives than those 
in Central Plains, Willow had been able to implement some instructional programs on its own. 
One such program, which involved a guarantee that all children at the school would be able to 
read by the end of second grade, had been initiated by the principal. However, in bringing the 
program before the faculty and asking them to decide as a group to implement it, he had produced 
strong teacher buy-in. As one teacher explained, “that was real accountability.” She continued: 
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It’s self-imposed, which was really nice, rather than an outside agency saying 
you will do this. I mean we decided we would do it. 

Just as at Poplar, teachers at Willow tended to credit the principal’s promotion of teacher buy- 
in and teacher-led decision-making as crucial to their sense of accountability. 



Limitations on Principal Influence 

It is important to note that not every principal in our study played a pivotal role in teachers’ 
understandings of and experiences with accountability. Principal leadership on accountability 
was more difficult to detect in both of our North City schools. This district placed strict 
limitations on principals’ autonomy. The presence of the Open Court coaches, for example, 
brought district-level supervision directly into the classroom, essentially superceding the principal 
as instructional leader. In addition, there were specific contextual factors at both North City 
schools that may have limited the principals’ impact on teachers’ experiences of accountability. 

At Maple, where the API was over 800, teachers were for the most part insulated from 
accountability pressures and left to make pedagogical and curricular decisions on their own. 
At Redwood, a core group of teachers actively opposed both accountability-related district 
reforms and the appointment of the new principal. Their efforts to become a “teacher-run” 
school and their open resistance to the principal likely impacted his ability to be an effective 
school leader. At both schools, teachers rarely mentioned their principals in the context of 
accountability, classroom instruction, student assessment, or professional development. How- 
ever, the principal at Redwood did say that he tried to “buffer teachers and other staff from 
the insanity that is downtown [i.e., the school district].” 

Our research found that these schools were the exception rather than the rule. In general, we 
found that principals played a key role in how individual schools and teachers experienced 
state -led accountability in California, and that the importance of their role should not be 
underestimated in future educational policy decisions. 
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Accountability and Equity 



A common goal of accountability reform nationwide, at both the state and federal levels, has 
been to draw attention to discrepancies in achievement across groups of students, based on 
such factors as race/ethnicity, language, and socioeconomic status, and to hold schools 
responsible for addressing any inequities. This chapter examines the challenges and possibilities 
of California’s accountability policy to address issues of equity, given the realities of educators’ 
assumptions and experiences within today’s school context. 

The state policymakers interviewed felt that a primary aim of California’s system of account- 
ability was to shine the light on inequities: “Explicitly the system sunshines the equity issue. 
The major motivation was to look at subgroups” (California Department of Education 
official). The system provides educators with data from the state’s annual standardized test, 
disaggregated by subgroups, with the assumption that educators will use that data to illumi- 
nate discrepancies and construct solutions in order to close gaps in achievement. State (and 
now federal) policies provide the mechanism in the data; teachers and administrators are 
expected to effect the change. A system of rewards and punishments is intended to provide 
further motivation for schools to close gaps. 

Disaggregated Data as a Mechanism for Change 

Policymakers hoped that the distribution of disaggregated test scores, both within the school 
and district contexts and in the public arena (through publication in local newspapers and 
the state education website), would create a heightened awareness among educators of gaps 
in student achievement. This in turn would spark a sense of accountability, as teachers and 
administrators would be motivated to adjust practices to better serve students and, ultimately, 
raise test scores. However, such an assumption does not take into account school-level chal- 
lenges of ensuring access to the data, skills to analyze the data, and most importantly, oppor- 
tunities to design and implement solutions to any gaps the data might reveal. Our research 
reveals that the path from test score publication and analysis to pedagogical change and 
academic improvement was not always smooth. 

Exposure to and use of subgroup data to influence school-level policies and practices were 
inconsistent or nonexistent at the sites we visited. District administrators and principals were 
generally familiar with the data; however, the same could not always be said for teachers. 
Some teachers had no knowledge that data existed in a disaggregated form. Others were 
aware of its existence, but unfamiliar with the details for their school. Most teachers were 
familiar with subgroup data only in the context of whether the school had met its subgroup 
targets that year. They knew, for example, that their Latino population had gone up by 30 
points, but that their Asian population had missed the target by 1 point. It was less common 
to hear teachers involved in an ongoing analysis of disaggregated data to inform instruction. 
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Teachers’ exposure to and use of subgroup data was often linked to principal involvement. 
For example, 70% of teachers interviewed at Poplar mentioned the use of subgroup data, as 
compared with only 10-25% of teachers at most other sites. Poplar’s teachers repeatedly 
described how their principal presented disaggregated test scores at staff meetings throughout 
the school year. Teachers at other sites, however, usually only encountered the subgroup data 
once a year, at the time of release, when the principal would post the school’s scores. 

Among those teachers who had seen the data, some felt it merely confirmed what they 
already knew about their students’ performance. Yet the majority of teachers who were 
exposed to the data did report an increased awareness of inequitable patterns of achievement. 
In some cases, those patterns came as a surprise, challenging educators’ assumptions about 
race and student achievement: 

I think what was interesting that came out with Maple’s [results] was that 
those kids who had scored in the lower 25 th percentile were not all African 
American; there were a fair number of Caucasian kids in there too. I think 
everyone kind of went, “Oh! We made this assumption and look, it isn’t 
necessarily true.” (Teacher, Maple) 

Occasionally, subgroup data analysis illuminated the need to find a balance in instructional 
focus between both low- and high-performing students: 

This last year we saw an increase in our English language learners and how 
they performed. But actually a decrease in our English-tw/)/ speakers. And 
we were trying to figure out what that could be attributed to. If it was that 
we were putting more energies towards English language development. 

(Teacher, Poplar) 

Exposure to subgroup data generally had the intended effect of increasing educators’ aware- 
ness of discrepancies in student achievement. What they did with that awareness, however, 
varied across sites. 

Only two of the schools visited responded with programmatic changes, creating after-school 
or Saturday programs for low-performing students. The principal at one of those schools 
explained how the subgroup data was useful in leveraging scarce funds for her Saturday 
program, as she could now document the need: 

It helps me, particularly with my parents and teachers... I would never be 
able to get my parents to focus in that way without the data. That [Saturday 
program] never would have passed through the PTA board or the teachers if 
we didn’t have the data to show the specific target group. (Principal, Maple) 

Unfortunately, awareness of discrepancies did not always lead to changes in practice. Most 
teachers expressed a sense of futility. They knew that inequities existed — it was the solutions 
that remained unclear. Frustrations stemmed from teachers feeling overwhelmed and lacking 
the resources and support to implement a solution. One teacher explained the challenges of 
addressing a low- achieving English Learner (EL) population at her school: 

We have felt for a couple of years with our increasing EL population, that 
there was no way classroom teachers were going to be able to, you know, 
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with all the other stuff that the classroom teacher has to do to bring that 
group along.... We don’t have the materials because we don’t have money in 
our budget.. ..We definitely try to do some things. I don’t think we’re satis- 
fied with what we’re able to do. (Teacher, Redwood) 

While some educators struggled to focus attention on a specific population despite limited 
resources, others simply resisted a group-specific focus in their teaching: 

We look at kids as individuals, not as Hispanic. Not as Asian kids, not as 
white kids. ...This disaggregated data gives us some information. Does it 
change what we do? No. And the reason it doesn’t change what we do is 
because what we do is based on individual kids, not on classifications of 
kids. And we bust our butts with Jose as much as we bust our butts with 
Ronell as much as we bust our butts with anybody else. (Aspen principal) 

In fact, it is significant to note that the majority of teachers and administrators interviewed 
insisted that they focused attention on students as individuals, not as members of a certain 
subgroup. While accountability policies highlight race and other subgroup categories as key 
student identifiers, educators insist that such categories do not factor into their teaching. 
Teachers resisted the policy’s intention to focus instruction based on students’ achievement 
by categories of race, language, or class, and not by individual. 

California’s accountability system employs subgroup data as a mechanism for change. The 
success of such a system is dependant upon several factors, including educators’ exposure 
to the data and their willingness and ability to analyze the data, their capacity to respond to 
the data with effective solutions, and an approach to teaching which emphasizes the catego- 
rization of students above individual differences. As our research demonstrates, such factors 
were not always in place to ensure the use of disaggregated data as an effective means to 
address inequities. 



Exposure to and use 
of subgroup data to 
influence school-level 
policies and practices 
was inconsistent or 
nonexistent. 



Roadblocks on the Path to Equity 

Educator capacity 

Despite educators’ insistence on a “color-blind” approach to instruction, our interviews 
revealed certain assumptions about student achievement as related to race, class, and culture 
that, unless challenged, may limit the ability of the state’s accountability policy to address 
issues of equity (Lipman, 1998). It was not uncommon, for example, for teachers to excuse 
low achievement among Latino and African American students by citing a lack of parent 
support, or a community culture that does not value education. A focus group discussion 
among teachers and administrators at Beech included the perception that African American 
students at that school were simply more interested in basketball than academics. The success 
of any accountability reform lies in the belief that all children can succeed. However, our 
research finds and other studies confirm, that not all educators may hold that belief, or more 
importantly, feel that they have the capacity to act on that belief (Kannapel et al, 1996). 

We heard a sense of futility among educators, a feeling that there was only so much improve- 
ment they could expect, given the demographics of their school: 
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We do a lot with what we have. And to me the natural consequence is 
they’re gonna improve and you’re going to reach a certain point. And 
then based on your demographics, you’re not going to improve anymore. 
(Principal, Aspen) 

The expectations of accountability policy were seen as unrealistic, given the enormous diversity 
of students. Several administrators suggested that progress should be measured in the context 
of their particular student population, echoed by analysts who argue that an accountability 
system must provide additional time and resources to high-poverty schools (Betts et al, 2000). 14 

This teacher felt frustrated that schools with a high-poverty student population faced the 
same sanctions as wealthier schools, with little or no support from the state: 

[Policymakers] are just putting pressure on saying, fix it or I shoot, you 
know. How? I don’t know, but you are going to have this test and you make 
sure they pass it or you are gone.... And it is ridiculous because it is not like 
we are given tools with which to help correct the problem. (Teacher, Aspen) 

Other researchers have noted that accountability policies do not typically provide sufficient 
guidance and resources to ensure a school’s capacity to improve (Fuhrman, 1999). 

Flaws in the system 

Educators across all sites discussed the challenges presented by their particular set of students, 
whether the result of racial and socioeconomic diversity or high percentages of EL or SPED 
students. Yet while most educators used demographics to explain student performance, it is 
important to note that they did not all use it as an excuse. The principal and teachers at 
Spruce, for example, articulated a “no excuses” attitude, striving to serve the needs of their 
students regardless of high rates of poverty and second language learners. Ultimately, educa- 
tors were frustrated by the system of accountability, and what many felt were its failures to 
provide solutions to the very real challenges of today’s diverse student populations. 

Teachers spoke a great deal about the mismatch between state and district-mandated cur- 
ricula and testing and the diversity of students, particularly EL and SPED, in their class- 
rooms. As noted earlier, the state’s annual standardized test was seen as an ineffective measure 
of learning for students who had not yet mastered the English language. The pacing plans for 
mandated math and reading programs were too fast, and teachers needed to supplement 
them with materials more appropriate for EL and SPED students. 15 The notion of having 
one curriculum meet the needs of all students seemed unrealistic. 

Educators were not convinced that accountability policies could address achievement gaps, 
pointing to flaws in the current system. They recognized that any effective solution would lie 
beyond simply reviewing test scores. Yet current policies and school practices do not always 
allow for a deeper investigation as to “why” inequities exist: 

But the reason why, those issues aren’t dealt with. So I think if you’re going 
to look at them [scores] and compare, they need to deal with the issues to 
why there is any discrepancy.... And so when they just say, ‘Well, teachers 
need to provide interventions,’ you’re still not dealing with why there is 
such a discrepancy. (Teacher, Maple) 
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The Fallout of the Accountability System 



Both the California and the federal No Child 
Left Behind accountability policies operate 
under the assumption that disaggregated 
achievement data is a mechanism to high- 
light inequities and close gaps. Moreover, 
state and federal sanctions are tied directly 
to a school's ability to meet performance 
targets by subgroup, and not simply school- 
wide. While such a focus on subgroups has 
increased educators' awareness, it occasion- 
ally results in a case of misguided attention. 

Willow Elementary School presents such a 
case. While still below the state target of 
800, Willow has made significant gains in 
overall student achievement, raising its API 
score from 557 in 2000 to 664 in 2005. The 
school's Latino and low SES subgroups have 
made considerable progress as well, moving 
from 459 to 599, and 493 to 628, respec- 
tively. Willow's other two subgroups, white 
and Asian students, have consistently out- 
performed their peers, with scores of 699 
and 780 respectively, in 2003. 

In 2002, Willow's Asian subgroup scored 741, 
yet missed their growth target by three 
points. A similar situation occurred the year 
before when the white students, Willow's 
second highest subgroup, improved but 
failed to meet their target by one point. 
NCLB legislation includes a sanction against 
any school that fails to meet a subgroup 
growth target two years in a row. Thus Willow 
was told it would have to send out a letter 
informing parents that the school was 
underperforming and that they could 
choose to transfer their children to a dif- 
ferent school, at Willow's expense. 

In many ways, Willow represents a school 
that has embraced the system of account- 
ability, adopting state and district standards 
and implementing a successful reading 
intervention program. The school had re- 
cently been portrayed as a success story in 
the local media and held up as a model for 
other struggling schools in the district. 



Understandably, the news of this letter came 
as a shock to administrators and teachers. 

Educators felt demoralized by a misdirected 
"low-performing" label, and frustrated by 
continued pressures from district, state, and 
federal policymakers despite improved test 
scores. As the school's principal remarked, "It 
just makes you feel like all your effort is for 
nothing." The principal was particularly con- 
cerned about how this would affect his staff: 

Well, I think it's going to most affect 
the teachers because we work so hard. 

Just like, all the pressure we experi- 
ence, we've totally come together as 
a team and we're feeling really good 
about ourselves, and then we get this 
news that we didn't do good enough. 

And all the other subgroups just did 
amazingly well, except the highest 
one didn't go up enough, you know? 

And because of that, we're being 
labeled a low-performing school. I 
can't even believe the idiocy of that. 

The educators we interviewed were acutely 
aware of the fallacy of the policy. The super- 
intendent described the situation as "absolutely 
ridiculous," calling this "the fallout of the ac- 
countability system." While attention to student 
performance by subgroups is a necessary 
step to addressing inequities in achievement, 
it is important to recognize that a singular 
focus may not allow for the complexities of 
school reform. The intentions of accountabil- 
ity reform had backfired; now the policy was 
drawing attention to the highest-perform- 
ing students. And in its haste to hold schools 
accountable for raising all students' achieve- 
ment, the policy left no room to recognize 
one school's efforts to close the gaps. 

In a further twist of irony, Willow was later in- 
formed that it would not actually have to send 
out the letter, following the state's recalcula- 
tion of API scores in 2003. With an adjustment 
of a few points, the school had met its sub- 
group targets and was now deemed a success. 
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Accountability 
policies do little to 
help schools move 
from that "low- 
performing" label to 
a place where all 
students meet 
proficiency targets. 


Interestingly, another teacher at Maple told us that the district had actually provided in- 
service training around issues of diversity, yet there was no follow-up or productive outcome: 

North City spent a lot of money doing this particular workshop. I think it 
was a three-day workshop and we were given a big binder in regards to this 
equity, you know racial relationship, how we see people, those sorts of 

things Then, where did it go? You know, there is no continuity. They 

spent so much money. So much money is wasted. (Teacher, Maple) 

While the district initially opened a space for dialogue around issues of diversity, it did not 
provide opportunities for ongoing discussions as to why inequities exist, and more impor- 
tantly, how schools could address them. The educators interviewed continually reminded us 
that quick fixes are not the solution to addressing issues of equity. 

Educators also raised concerns about a policy based on designating students into subgroups. 
While they supported a policy that required improvement for all students, many worried that 
the policy’s focus on racial subgroups might lead to negative stereotyping or misguided atten- 
tion to certain groups of students: 16 

I think there’s some danger in that because teachers will look at that and say, 

“Those goddam whatever it is, they screwed our whole school.” You know? 

You can look at that and it can create more problems. . ..Or it can be used as 
a justification for the stereotypes that people already have, or “Yea, if it 
weren’t for those whatevers, we would be doing fine.” (Principal, Aspen) 

Others felt the designations were inaccurate or did not take into consideration within-group 
diversity. A racial subgroup designation of “white,” for example, might not capture the com- 
plexities of language, class, and ethnicity within that group: 

You’re talking about white, but white in our school could mean Bosnian, it 
could mean born here in Tech Valley, it could be Indian. And all of those 
have mitigating factors. We could be talking about differences of poverty, in 
education level,. . .differences in language skills. ...Are we talking equal when 
we’re talking about a child who lives in Tech Valley versus a child who’s 
brand new from India, Pakistan or some other part of the world? They’re 
not equal and yet they’re both white. (Teacher, Willow) 

Concerns about language, which cut across racial subgroups, were echoed at all of our sites. 
Yet during the time of our interviews, California’s system of accountability did not include 
English language learners as a subgroup. 

Due to NCLB requirements, California now includes EL and SPED subgroup designations, 
beginning with the 2003-2004 achievement data. 17 With the addition of these categories 
comes further incidents of overlapping subgroup designations. It would not be uncommon, 
for example, for a student in California to fall into the Latino, socioeconomically disadvan- 
taged, and EL subgroups. This has brought an additional layer of complexity to the system of 
accountability, and in some cases, the potential for additional sanctions. 

The inclusion of EL and SPED subgroups is laudable in its effort to encourage those students 
to work toward the same high levels of proficiency expected of other students. Elowever, with 
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an expanding list of subgroup categories comes increased possibilities for schools to receive a 
“low-performing” label. Recent analysis of California’s achievement data suggests that schools 
with three or more subgroups have a more difficult time meeting state and federal account- 
ability requirements, as they face more “trip wires” in the system (Novak & Fuller, 2003). 
Furthermore, early research on the impact of state subgroup systems finds no evidence of 
improved minority student achievement as compared to state systems without subgroup tar- 
gets (Kane & Staiger, 2002). 

These issues do raise questions as to the purpose of state and federal accountability policies. 
At this point, such policies serve the purpose of identifying and penalizing schools that fail to 
meet growth targets. The focus is on the outcomes, not the processes. Accountability policies 
do little to help schools move from that “low-performing” label to a place where all students 
meet proficiency targets, regardless of the diversity of the school. 

Educator Suggestions to Address Equity issues 

Given educators’ critiques of current accountability policies, and the challenges they face in 
today’s classrooms, what is the potential for a system of accountability to facilitate the 
closing of achievement gaps? Some educators say efforts to find solutions by illuminating 
inequities are futile: 

I suppose if we could get the answers, that’d be great, but what if we couldn’t? 

What if it really just meant, “Sorry, they don’t do as well.” (Teacher, Beech) 

Others argue that many of the factors that contribute to low-achievement are beyond the 
school’s control, including student demographics and family values: 

Yeah, I mean we talk about it a lot, but it boils down to the same thing.... it’s 
what they see at home, it’s their values and it’s their beliefs. There’s really 
nothing you can do . . . because it’s out of our hands. (Teacher, Aspen) 

It is especially important to note that for many educators, the goal of policymakers to expose 
inequities and motivate schools to close gaps went largely unrecognized. The link between 
the system of accountability and the closing of achievement gaps was unclear. For many, the 
end result of the system was simply to expose school failures or successes. The potential for a 
system of accountability to address inequities was often not considered at all or else not 
seen as feasible. 

Yet amidst the challenges and critiques, our conversations with educators did reveal aspects of 
the system that show potential to address issues of equity. The best example of this was the 
curriculum standards component. The majority of teachers interviewed saw standards as a 
practical tool to ensure equal opportunities, so that students in poorer districts received the 
same quality of education as students in wealthier districts. The uniformity of standards 
meant that all students would, in theory, be exposed to the same material and be held to the 
same expectations, regardless of their background or school characteristics. 

Educators also spoke about the effectiveness of data to address equity, if used in a construc- 
tive, creative approach at the school level: 
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You can look at it and say, “Why is this so?” instead of taking that information 
and saying, “You’re not doing a good job. These scores are low. Bring them 
up.” There’s a difference between taking information and using it to try to 
determine why it’s so and [seeing] if something can be done. (Teacher, Aspen) 

For example, as mentioned in Chapter 2, the teachers and principal at Poplar relied on a 
series of ongoing assessments to track student achievement throughout the year. Broken down 
by subgroups, the data from these more frequent, less “high-stakes” tests allowed teachers to 
note any gaps and adjust instruction in a timely manner. A few teachers also suggested that it 
might be more effective to look at data tracked over time by both student and teacher. The 
student data demonstrate patterns of low-achievement, perhaps revealing that a school’s Latino 
population is underperforming. The teacher data might take that a step further by pointing 
to effective instructional solutions if, for example, one teacher consistently produces higher- 
achieving Latino students over several years. 

Teachers and administrators also offered suggestions to improve the existing system’s efforts 
to address inequities. They emphasized the need for a more holistic approach to equity, incor- 
porating institutional and community support. The Central Plains superintendent believed 
that a more equitable system “really depends on the instructional leader, the principal,. . . the 
superintendent, and the local board and the local culture in the school district.” 

She recognized that the teacher ultimately held the power to make a difference: 

If they’re left alone in the isolation of their classroom. . . and there isn’t some- 
one who is saying, we will improve all children,. . .then there won’t be the 
real, true equity change we need (Superintendent, Central Plains). 

Finally, educators called for an expanded definition of equity, reminding us that such 
considerations must include not only student achievement, but also equality of resources and 
funding within schools and across districts: 

The accountability is placed on the wrong place. They need to look more at 
the... economic equity, and once we work on that element, then teachers 
and parents and politicians, you know, all of us, can work together to really 
help the students (Teacher, Maple). 

Equity for students can not be achieved without an infusion of resources, both material and 
non-material, to those neediest schools and districts. Some argue that the state needs to be 
held to “opportunity to learn” standards that would ensure adequate resources to all schools 
(Koski, 2002; Winfield & Woodard, 1994). 18 

While California has seen a slight narrowing of the achievement gap in elementary schools, it 
is still too early to determine whether this trend is sustainable. Research on similar systems of 
accountability in other states does not support the notion that a system of testing and rewards 
and punishments can ensure equitable outcomes, and may actually produce negative out- 
comes for students (Diamond & Spillane, 2002; Flaney, 2000; Madaus & Clarke, 2001). 
The theory of action behind accountability systems often falls short, as most states do not 
provide the necessary tools to help schools move from that initial awareness of the problems 
to a realization of solutions (Furhman, 1999, Massell, 1998). 
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In short, the issue is one of school capacity. Schools often lack the resources, both financial 
and intellectual, to close achievement gaps, and accountability policies have failed to address 
this issue. Higher-performing schools, for example, are more likely to use data to improve 
instruction and student learning (Diamond & Spillane, 2002). Lower-performing schools 
may lack the capacity to analyze data and implement an appropriate pedagogical or curricular 
response to low achievement. 

The current system is based largely on negative motivation. Educators have seen few positive 
“carrots,” especially within the context of California’s fiscal crisis. State and federal policies 
need to focus more on the successes, especially among those schools that are outperforming 
expectations. What little assistance that does exist should be continued, and expanded if 
possible, in an effort to build school capacity. For example, professional development and 
other “categorical” funds can be focused on building educators’ capacity to analyze data to 
influence instruction. District and school-level administrators need to engage teachers 
in the analysis of disaggregated data, as a means to recognize and address inequitable 
patterns of achievement. 

As the teachers and administrators interviewed remind us, it is not enough to simply point to 
the problems; the challenge of raising achievement for all students lies in building educators’ 
capacity to respond to that awareness. Educators need to feel a sense of possibility, not futility. 
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Policy Discussion 



As the voices of teachers and administrators bring to life, educators did not always respond to 
accountability according to the assumptions or intentions of the policy. However, their 
responses did not merely reflect a resistance to the system of accountability, but also an 
acknowledgment of its complexity. In spite of their critiques of the system, teachers and 
administrators also recognized its benefits. 

One noticeable result of recent accountability reforms is an increased awareness of issues 
surrounding student achievement, in large part the result of the API system and subsequent 
publication of school test scores and rankings. As one district administrator remarked, educa- 
tors can now “really begin, if we are willing to, to discuss those issues of equity and opportu- 
nity and access in a different way.” (Superintendent, Central Plains) 

Teachers also told us that they considered the state curriculum standards to be a useful tool, 
providing teachers with focus and ensuring uniformity across the state. There was evidence 
that standards encouraged teachers to hold their students to higher expectations. Further- 
more, California’s recent efforts to align its annual test with standards may provide educators 
with further understanding of the ways to enhance student learning, as curricular efforts will 
now be more directly reflected in students’ test results. 

Yet while educators believed that California’s system of accountability contained worthwhile 
components, many felt it was heavy on regulations and short on solutions. 

A primary goal of ERAP was to uncover the realities of school-level implementation in light 
of certain assumptions underlying California’s accountability system. One key assumption is 
the notion that the school is the primary locus of change, teachers are the primary actors of 
that change, and the expected outcome is improved instruction resulting in higher student 
achievement. Another assumption is that teachers will be motivated to enact change in 
response to a system of external rewards and punishments. 

Our interviews with teachers, principals, and district leaders remind us that putting policy 
into practice is a far more complex endeavor than setting policy guidelines. While the school 
may be the ultimate locus of change, it cannot be considered a solitary unit. The school is 
situated within a district, with its own set of policies and influences. Indeed, as discussed in 
Chapter 3, the district is often a source of additional accountability policies and pressures 
that may reinforce or complicate state policy. 

Similarly, teachers are not the sole actors within a school, but instead work in relationship with 
school administrators and students towards the goal of student learning. As discussed in 
Chapter 4, many educators were concerned that the state policy placed a disproportionate amount 
of responsibility on teachers, without taking into account the impact of other factors including 
student diversity, principal and district leadership, and capacity for improving student outcomes. 
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California’s accountability policy not only assumes that teachers will be in a position to enact 
change, but that they will be motivated to do so by a system of rewards, such as the Governor’s 
Performance Awards program, and punishments, ranging from a low-performing school 
label to the threat of state takeover. Our research suggests that such mechanisms did not have 
a significant effect on teacher motivation, in large part because the promises or threats of the 
policy did not seem real. The awards programs disappeared as a result of state budget cuts 
before many teachers saw a reward. Those who received the awards often felt they were not an 
appropriate gesture for their efforts, nor was it a motivating factor. Likewise, few educators 
felt that the state was in any position to take over a school, much less do a better job. Teachers 
reminded us that pressure and public shaming are not effective means to motivate teachers to 
improve instruction, and instead may lead to negative or unintended consequences for 
student learning and professional satisfaction. 

California’s accountability policy was also grounded in the assumption that test score data, 
disaggregated by race/ethnicity and socioeconomic status, would act as a further mechanism 
for change, specifically addressing inequities in student achievement. However, as discussed 
in Chapter 6, such an assumption did not take into account school-level challenges of ensur- 
ing access to the data, skills to analyze the data, and most importantly, opportunities to 
design and implement solutions to any gaps the data might reveal. 

Policy Considerations 

Findings from the Educator Responses to Accountability Project have helped to amplify teach- 
ers’ and administrators’ voices in education policy discussions, in an effort to illuminate the 
benefits and challenges of the system, and to expand policymakers’ understanding of 
accountability at the school level. Our conversations with educators also reveal aspects of the 
system, at both the state and district levels, that show potential for improvement. 

Ensure coherency and consistency in state accountability policy 

Educators expressed initial confusion over the alignment of standards with testing, the quali- 
fications for rewards, and the efficacy of interventions. Educators also expressed confusion 
about the changing nature of the system of accountability, including the calculation of API 
scores with different tests from year to year, and the availability and distribution of rewards. 
For example, the state’s initial system of rewards and sanctions has been shifted to essentially 
a system of sanctions. In order to support and encourage schools, the state should consider 
alternative positive incentives, in an effort to maintain consistency with the system’s original 
intention. A reform strategy relying on punishments alone can not provide the motivation to 
sustain improvement efforts in schools. 

Although PSAA has been extremely successful in bringing attention to issues of student achieve- 
ment, its clarity and understanding are essential to any sustained impact. As federal mandates 
continue to overlap with California’s system, it is increasingly important that the state make 
efforts to explain accountability policies to educators, parents, and students. The state should 
investigate ways to improve educators’ understanding of the policy’s guidelines, as a first step 
to ensuring their capacity to meet its requirements. 
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Focus on building and sustaining educator capacity at all levels 

California’s accountability policy is based upon the assumption that most teachers and 
administrators hold the capacity to improve performance and address gaps in student achieve- 
ment. However, as our conversations revealed, many educators felt they did not possess the 
proper tools, including time, resources, and administrative support, to systematically address 
gaps in achievement. State and district leaders should consider ways to ensure that educators 
have the capacity to improve instruction and student learning. 

Professional development is a key factor in building and sustaining educator capacity. 
Educators often felt frustrated by a lack of control over their professional development time 
and funding. The implementation of district-mandated curriculum packages, for example, 
included district-mandated trainings, often leaving teachers and principals with few 
opportunities for collaboration and development in other areas. Training in a curriculum 
package may provide only one aspect of educators’ development. Moreover, teachers’ com- 
mitment to improvement necessitates a role in decisions around the timing and topics of 
professional development. 

Together, state and district leaders should explore additional ways to allow schools to address 
their particular needs and interests. The state might develop guidelines around professional 
development, allowing for district- and school-level flexibility while ensuring adequate 
capacity-building opportunities to meet the demands of accountability. 

In particular, professional development should address the use of data. Our research demon- 
strates that educators lack exposure to data as well as the time and skills to analyze data to 
inform instruction. As a result, educators are unable to achieve a primary goal of accountabil- 
ity, namely to address inequities through the analysis of disaggregated data. Professional 
development could provide opportunities to raise educators’ awareness of the availability of 
data and then provide the skills to decode it. Trainings might focus, for example, on the 
usefulness of disaggregated data to reveal trends in student achievement. Furthermore, trainings 
might demonstrate ways to link data to pedagogical responses. 

Finally, the state should consider the implications of holding all schools to the same expecta- 
tions while failing to address existing disparities in resources, materials, and funding, within 
and across districts. The state cannot afford to continually label schools as failing without 
providing necessary support. The state’s High Priority Schools Grant Program represents an 
effort to direct resources and support to those schools that need them the most. Fikewise, the 
states’ consideration of “Opportunity To Fearn” indicators is an important first step in ensur- 
ing that all schools are provided with adequate essential resources . 19 

Recognize the integral role districts play in accountability 

Currently, accountability policy assumes that individual schools are the primary locus of 
reform, focusing on school API scores, student subgroup data, and classroom curriculum and 
instruction. While school-based reform is essential, local districts play a large role in deter- 
mining how individual schools interpret and implement various mandates. 



While educators 
believed that 
California's system 
of accountability 
contained worthwhile 
components, many 
felt it was heavy on 
regulations and short 
on solutions. 



The state should recognize district influence in the implementation of accountability policy. 
As accountability demands continue to place pressure on schools, it is increasingly vital that 
schools are able to turn to their districts for support and guidance. Our research shows 
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California's 
accountability policy 
does not take into 
account challenges 
of ensuring teachers 
access to test data, 
skills to analyze it, 
and opportunities to 
implement solutions. 



that districts, through various programs and policies, have the power to facilitate school 
improvement. Teachers, for example, appreciated district efforts to clarify the sometimes 
cumbersome state standards into “essential” standards. The state can provide a means to 
research and disseminate similar examples of successful district responses to accountability. 

Due to federal NCLB requirements, the state is now considering ways to hold districts 
accountable for the performance of their schools. Educators interviewed agreed that the state 
should provide a ranking or score for districts, similar to the Academic Performance Index 
(API) system for schools. Such a system should include a special focus on districts with the 
lowest-performing schools that have shown little to no signs of improvement. 

Make strategic changes to the assessment and data components 

While educators appreciated the states efforts to align the annual test with curriculum standards, 
they challenged the utility and fairness of relying on one assessment to measure student 
achievement. In order to piece together a more coherent picture of student learning, state and 
district leaders might consider the use of ongoing, standards-based diagnostic assessments. 
Our research demonstrates that the success of such assessments is directly linked to teachers’ 
involvement in decision-making processes around adoption and implementation. Moreover, 
it is important to note that this testing should not carry any rewards or sanctions, so as to not 
add to the “testing burden.” Instead, such tests would provide more immediate feedback, 
helping teachers to focus instruction and facilitate student learning. 

Data from diagnostic testing would also supplement data from the state’s annual standard- 
ized test. Teachers interviewed critiqued the timing and alignment of standardized test data. 
Many felt that test results were delivered too late to be of use with their students during the 
current academic year. Although data did help illuminate achievement gaps, it failed to aid 
teachers in making valuable instructional changes during the course of the school year. On- 
going assessments would provide teachers with more timely information about their stu- 
dents’ learning. It is too soon to tell whether data from the standards-aligned CAT-6 test will 
meet teachers’ diagnostic needs. If not, state and district leaders should consider alternative 
diagnostic measures, in conjunction with efforts to expand educators’ capacity to utilize such data. 



Conclusion 

This report provides policymakers with an understanding of educators’ experiences with 
California’s public school accountability system. As state and federal policies continue to 
place demands upon schools and districts, it is essential that policymakers continue to listen 
to teachers and administrators. ERAP findings provide the impetus for an ongoing dialogue 
between and among policymakers and educators. 

One aspect of that dialogue must include consideration of how best to meet the goals of 
accountability, namely to enhance student learning and close achievement gaps. In reflecting 
upon the successes and challenges of California’s system of accountability, educators inevita- 
bly asked, “Where do we go from here?” There was little disagreement with the policy’s 
assumptions, for example, that many of California’s public schools are in need of improve- 
ment, or that teachers and administrators should be held accountable for their students’ 
learning. Educators especially appreciated the system’s efforts to hold all students to the same 
high expectations. 
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What was less clear to educators was how to improve those schools, and how to ensure high 
levels of accountability and student achievement. Our research provides a glimpse into those 
next steps, revealing some of the more creative ways schools and districts are choosing to 
respond to or supplement accountability mandates. This is only a first step in a necessary 
investigation and sharing of knowledge — knowledge between policymakers and educators, 
and among teachers and administrators. All could benefit from further discussion of effective 
responses to accountability; stories of a teacher using data to transform pedagogy, for 
example, or a district’s creative approach to improving its lowest-performing schools. 

Above all, California’s “next steps” in public school reform must include a more holistic 
approach to accountability. Policymakers and educators might consider whether the current 
system of accountability, with its singular focus on teachers and school-level change, is the 
most effective means to improve student achievement. As our research in schools demon- 
strates, the district plays an integral role in school and classroom practices. Likewise, parents 
and students contribute to the success of a school. A more inclusive approach to accountability, 
one that recognizes the influences of the broader community, would better support teachers 
in their efforts to improve California’s schools. 
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Endnotes 

1 See Appendix 1 for overview of California’s system of 
accountability. 

2 At the time of our research, NCLB requirements had 
not yet been integrated into the state’s accountability 
system. Therefore, our conversations with educators 
focused on the state system, with some educators of- 
fering early perspectives on NCLB. 

3 See Appendix 2 for details on ERAP schools. 

4 See Appendix 4 for interview protocols for teachers, 
principals, and superintendents. 

5 See “State Policymakers’ Views on Accountability” in 
Chapter 1 for themes that emerged from our conver- 
sations with policymakers. 

6 As discussed in Chapter 3, the Tech Valley district 
chose to distill the state standards into “essential” dis- 
trict standards, in an effort to provide teachers with 
the benefits of focus and direction in a more manage- 
able system. 

7 Under the new Instructional Materials Funding Re- 
alignment Program (IMFRP), which began in 2002- 
2003, all districts must adopt state-approved curricu- 
lum materials in math, reading, science and social 
studies within 24 months after their adoption by the 
State Board of Education. 

8 In 2001-2002, seven schools in the Central Plains 
district met their growth target, three schools showed 
an increase in their API scores but did not meet their 
growth targets, and five schools maintained or showed 
a decrease in API scores and did not meet growth 
targets. Both schools in our sample showed a decrease 
in their API scores and did not meet their growth 
targets. These two schools raised their API scores and 
met growth targets the following year. 

9 The usefulness of these programs can also be attrib- 
uted to the timeliness of their data, as discussed in 
“Flow Ongoing Standards-Based Assessment En- 
hances Accountability” in Chapter 2. 

10 See Chapter 5 for further discussion of the Central 
Plains principals. 



1 1 Science was added to the STAR testing program dur- 
ing the 2002-2003 school year. 

12 See “The Fallout of the Accountability System” in 
Chapter 6 for further information. 

13 See “Flow Ongoing Standards-Based Assessment 
Enhances Accountability” in Chapter 2 for fur- 
ther discussion. 

14 California did provide additional resources to low- 
performing schools through the II/USP program. 
However, an extensive evaluation of the program did 
not reveal significant benefits to those schools enrolled 
in II/USP as compared with similar schools outside 
the program (O’Day & Bitter, 2003). 

15 The programs in use at many schools are required to 
have an EL component, yet it was clear that educa- 
tors did not find that adequate. Interestingly, the state 
also includes a reading intervention program for EL 
students on its list of state-adopted curricula. Yet this 
program was mentioned in only a handful of our in- 
terviews with educators across the state. Teachers and 
administrators were either unaware of or did not have 
access to the materials, and thus were left with the 
sense that the state was not doing its job of providing 
teachers access to the most effective materials for their 
diverse range of students. 

16 See “The Fallout of the Accountability System” in 
Chapter 6 for an example of “misguided attention.” 

17 Neither California nor NCLB accountability policies 
designate gender as a subgroup growth target. 

18 California is currently involved in a legal dispute 
around issues of adequacy. The state is also in the pro- 
cess of appointing a commission to look into the de- 
velopment of Opportunity to Learn standards. 

19 California’s Quality Education Commission has been 
charged with developing a quality education model 
which would ensure the necessary resources for stu- 
dents to meet the state standards. The future of this 
commission is uncertain, given recent political shifts. 
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Appendix 1 

PSAA Overview & Evolution 





1999-2000 


2002-2003* 


ACADEMIC 

PERFORMANCE 

INDEX 


1. Composite based on results on the 
Stanford 9 test, a normed 
referenced test. 


1. Composite based on the following: 
a) California Achievement Test (CAT-6) 
a normed reference test (20%). 




2. Weight given Sat-9 content areas: 
math (40%), reading (30%), language 
(15%), and spelling (15%). 


b) California Standards Test (CST) in 
ELA and Math (80%). For grades 4 
and 8 a writing assessment is included. 


STANDARDS 


Although SBE adopted standards for 
ELA, mathematics, social science/history, 
and science in 1997, they are not re- 
flected in API testing. 


CST reflects content standards. 


REWARDS 


1 Governor's Performance Award 
Program (GPAP) 

$96 million is provided for schools that 
meet or exceed their API growth targets 
overall and for significant subgroups. 


Rewards program did not receive any 
funding after Fall 2000. 




2 Certificated Staff Performance 
incentive Act 

$50 million is provided for one-time 
bonuses to teachers and other staff 
in underachieving schools that 
significantly improve. 






3. Non monetary awards 




INTERVENTIONS 


immediate intervention/ 
underperforming Schools Program 
(ll/USP) 

$96 million is provided to support an 
initial group of 430 schools that volun- 
teered and were selected. Schools must 
be in lower five API deciles. 


ll/USP program did not receive any 
funding after Spring 2003. 

High Priority Schools Grant 
Program (HPSGP) 

It replaced ll/USP, and provides 
assistance to lowest performing 
schools (decile 1), regardless of 
progress. 


SANCTIONS 


Schools in the intervention program that 
do not meet growth targets or show 
significant growth over time will be 
subject to local interventions and 
eventually state sanctions. 


As of 2003, the state had not 
subjected any schools to state 
sanctions. 



* ERAP site visits occurred during this school year. 
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Appendix 2 

ERAP Schools (2002-2003) 



School 


District 


Location 


Enrollment 


Ethnic Distribution 


API 

Subgroup 

Scores 


State/ Sim 
Rank 


API 

Overall 

Score 


Aspen 


Tech Valley 


Suburban 


386 


6% Afr. American 


NA 


7/10 


787 










17% Asian 


870 














36% Hispanic 


705 














20% White 


869 














69% F/R Lunch 


731 














37% ELL 


NA 






Beech 


South City 


Urban 


313 


38% Afr. American 


741 


5/7 


720 










2% Asian 


NA 














44% Hispanic 


677 














16% White 


NA 














71% F/R Lunch 


695 














16% ELL 


NA 






Maple 


North City 


Urban 


432 


21% Afr. American 


785 


9/9 


879 










5% Asian 


NA 














5% Hispanic 


NA 














60% White 


920 














10% F/R Lunch 


NA 














2% ELL 


NA 






Poplar 


Central Plains 


Rural 


292 


.5% Afr. American 


NA 


7/7 


780 










1% Asian 


NA 














30% Hispanic 


675 














69% White 


824 














25% F/R Lunch 


642 














12% ELL 


NA 






Redwood 


North City 


Urban 


334 


70% Afr. American 


693 


5/6 


708 










8% Asian 


NA 














12% Hispanic 


NA 














7% White 


NA 














67% F/R Lunch 


688 














14% ELL 


NA 






Spruce 


Central Plains 


Rural 


141 


1% Afr. American 


NA 


1/NA 


580 










2% Asian 


NA 














82% Hispanic 


557 














11% White 


NA 














100 F/R Lunch 


570 














67% ELL 


NA 






Sycamore 


South City 


Urban 


1206 


1% Afr. American 


NA 


3/9 


663 










.1% Asian 


NA 














96% Hispanic 


658 














2% White 


NA 














94 % F/R Lunch 


663 














80% ELL 


NA 






Willow 


Tech Valley 


Suburban 


439 


5% Afr. American 


NA 


4/3 


664 










18% Asian 


780 














50% Hispanic 


599 














19% White 


699 














83% F/R Lunch 


628 














59% ELL 


NA 
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Appendix 3 

Trend of API Scores 



1000 - 




300 - 



1999 2000 2001 2002 2003 

School Year 
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Appendix 4 
interview Protocols 

Teacher Interview Protocol 

A. Teacher Experience and Background 

■ Please fill me in briefly on your background. 

■ What do I need to know about you as a teacher? 

B. School Context 

■ How would you describe this school to colleagues in 
another school? 

■ Please tell me about your students this year? 

■ How are they similar or different from other years? 

■ What are the most pressing issues you currently face in 
the classroom? 

■ How are you addressing them? 

■ What are the big issues or pressing concerns currently 
facing the school? 

■ Do you feel you have a part to play in addressing those 
issues/concerns? If so, then how? 

C. School Reform 

A lot of people in California are talking about how to 
improve the schools. I now want to talk with you about 
recent changes and how these changes may be affecting 
your work. 

■ What are the most significant reform changes you as a 
teacher and the school have faced in the last few years? 

■ Have those changes affected your work? If so, how? 

■ The main focus of our study is how educators are 
responding to California’s system of accountability. 

Can you tell me how accountability plays out over the 
course of the school year? 

■ Has the school structured activities or professional 
development in response to accountability? 

■ What does test month look like in your school? 

■ Are you aware of your school’s API score, subgroup 
data, or its ranks? What are they? 

■ Do you feel this system of scores and ranks provides an 
accurate representation of your school? Why or why 
not? Are these scores and ranks useful? 

■ How do this school’s API score, subgroup data, or 
ranks affect the work of the school? 

■ How does the school use the data? 



D. Equity 

Some people hope that the system of accountability will 

promote higher achievement for all students. 

■ Does your school use the API or subgroup data to 
promote equitable achievement? 

■ Do you think the system of accountability is addressing 
issues of equity for kids throughout the state? 

■ Are there other ways that the school responds to 
student diversity (like race/ethnicity, class, or language 
issues)? 

E. Pedagogy/Professionalism 

■ Has the system of accountability affected your sense of 
what it means to be a teacher? If so, how? 

■ Have your goals for your students been affected by the 
system of accountability? If so, how? 

■ Has your teaching practice changed as a result of the 
system of accountability? If so, how? 

■ Has the content of your teaching changed as a result of 
accountability? If so, how? 

F. Wrap-up 

■ Where do you see yourself professionally in the next 
5-7 years? 

■ Is there anything else you would like to add before we 
end, in particular any recommendations about the 
system of accountability for policymakers? 

Principal Interview Protocol 

A. Principal Experience/Role as Principal 

■ Please fill me in briefly on your background, and how 
you came to be a principal at this school. 

■ How would you describe your philosophy as a principal? 

B. School Context 

■ How would you describe this school to colleagues in 
another school? 

■ Please tell me about the students and parents at this school. 

■ What are the most significant reform changes you as a 
principal and the school have faced in the last few 
years? 

■ Have those changes affected your work? If so, how? 

C. Philosophy and Practices around Accountability 

■ The main focus of our study is how educators are 
responding to California’s system of accountability. 
What is your philosophy on accountability? 
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■ Has California’s system of accountability influenced 
your philosophies and practices as a principal? If so, 
how? 

■ How do you implement your philosophy? What are 
the tools or techniques that you use? 

■ How do your teachers respond to those tools? 

■ Are you aware of your school’s API score, subgroup 
data, and its ranks? What are they? 

■ Do you feel they are an accurate representation of your 
school? Why or why not? 

■ How does this school’s API score or rank affect the 
work of the school? Your work as a principal? 

■ How does the school use the data? 

D. Equity 

Some people hope that the system of accountability will 

promote higher achievement for all students. 

■ Does your school use the API or subgroup data to 
promote equitable achievement? 

■ Do you think the system of accountability is addressing 
issues of equity for kids throughout the state? 

■ Are there other ways that the school responds to student 
diversity (like race/ethnicity, class, or language issues)? 

E. Their "Place in the Chain" 

■ What are your district’s expectations in terms of 
accountability for your school? For you as a principal? 

■ Do you receive support or resources from your district 
to meet those expectations? 

■ How do you work with teachers to meet those 
expectations? 

■ Are their ways for you and your teachers to voice input 
about accountability to the district? 

F. Wrap-up 

■ Where do you see yourself professionally in the next 
5-7 years? 

■ Is there anything else you would like to add before we 
end, in particular any recommendations about the 
system of accountability for policymakers? 

Superintendent/District Administrator 
Protocol 

A. Superintendent Experience/ Role as Superintendent 

■ Please fill me in briefly on your background, and how 
you came to be a superintendent in this district. 



■ How would you describe your philosophy as a 
superintendent? 

B. District Context 

■ Please tell me about the students and parents this 
district. And the teachers? 

■ What are the most significant reform changes the 
district has faced in the last few years? 

■ Have those changes affected your work? If so, how? 

C. Philosophy & Practices around Accountability 

■ The main focus of our study is how educators are 
responding to California’s system of accountability. 
What is your philosophy on accountability? 

■ How do you implement your philosophy? What are 
the tools or techniques that you use? 

■ How do your principals respond to those tools? 

■ Do you think the current system of accountability is 
addressing issues of equity for students? If so, how? 

■ Are you aware of our two focus schools’ API score, 
subgroup data, and ranks? What are they? 

■ Do you feel they are an accurate representation of 
those schools? Why or why not? 

■ How does those schools’ API score, subgroup data, or 
ranks affect the work of the district? 

■ How does the district use the data? Is it used 
specifically to address issues of equity? 

D. Their "Place in the Chain" 

■ What are your expectations in terms of accountability 
for your district? And where are those expectations 
coming from? 

■ Do you receive support or resources from the state to 
meet those expectations? 

■ How do you work with principals and teachers to meet 
those expectations? 

■ Are there ways for your principals and teachers to voice 
input about accountability to the district or the state? 

E. Wrap-up 

■ Where do you see yourself professionally in the next 
5-7 years? 

■ Is there anything else you would like to add before we end? 
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